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DESCRIPTION 



Image Processing Appara'bus and. Metiiod. 

5 TECHNICAL FIELD ' 

The present invention relates to an image processing apparatus 
and a method, and in particular to an image processing apparatus 
and a method preferably applicable to conversion of a wide 
10 dynamic-range image having a dynamic range of pixel values wider 
than the normal one to a narrow dynamic-range image having a narrower 
dynamic range of pixel values, and to enhancement of contrast. 

BACKGROUND ART 



(Charge Coupled Device) and CMOS (Complementary Metal-Oxide 
Semiconductor) have been widely used for the imaging instruments 
such as video cameras and still cameras; and optical measurement 
20 apparatuses such as component inspection apparatus used in FA 
(Factory Automation) and optical measurement apparatuses such as 
electronic fiberscope used in ME (Medical Electronics) . 

In recent years, there are proposed a large number of techniques 
25 for obtaining an image having a wide dynamic-range (referred to 
as ^^wide-DR image", hereinafter) of pixel values in comparison with 
that of optical film photograph using these solid-state imaging 
elements . 
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Conventionally, solid-state imaging elements such as CCD 
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On the other hand, display apparatuses for displaying moving 
image and still image such as CRT (Cathode Ray Tube) and LCD (Liquid 
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Crystal Display), projection apparatuses such as projector, and 
various printing apparatuses are not yet widened in their supportable 
dynamic range of pixel values at present, and have only a limited 
range of supportable luminance grayscale. Hence the present status 
5 is that, even a wide-DE image should have successfully pictured, 
there are no apparatuses capable of displaying, projecting or 
printing the image as it is obtained. 

Hence there is a need for a technique (referred to as ^^grayscale 
10 compression technique", hereinafter) by which the dynamic range 
of pixel values of the wide-DR image is narrowed, or in other words, 
the luminance grayscale is compressed, so as to produce an image 
(referred to as ^'narrow-DR image'', hereinafter) adapted to the 
dynamic range of the display apparatuses and so forth. 

The following paragraphs will explain a 

conventionally-proposed grayscale compression technique. The 
grayscale compression technique can simply be achieved by 
re-assigning the grayscale of pixel values of the wide-DR image 
20 so as to be suited to grayscale of a narrower dynamic range supportable 
by the display apparatuses or the like. 

However, as described in the above, a uniform re-assignment 
of the grayscale of pixel values of the wide-DR image simply to 

25 the narrow dynamic-range only results in a reduced luminance 
variation of the image as a whole, and consequently in conversion 
into a poorly-looking image with a degraded contrast. There are 
conventionally proposed some grayscale compression techniques 
capable of suppressing the loss in contrast. Three grayscale 

30 compression techniques ever proposed will be explained below. 
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A technique which can be exemplified as a first grayscale 
compression technique relates to an adaptive determination of a 
grayscale redistribution rule based on a histogram of luminance 
of an input wide-DR image (more specifically, calculation of a 
5 grayscale conversion curve based on a histogram of an input image) • 
The first grayscale compression technique is on the premise that 
a principal subject in an image has a large ratio of occupational 
area, and is to determine a grayscale conversion curve so as to 
assign the grayscale as much as possible to a luminance value at 
10 around a peak in the histogram, to thereby suppress lowering in 
the contrast of at least the principal subject. 

It is, however, difficult to obtain a satisfactory result in 
every circumference only by an effort based on the grayscale 
15 assignment. In an exemplary case where an image has a plurality 
of principal subjects and has a background with a uniform luminance 
and a relatively wide area (e.g., blue sky), the subjects often 
fail in obtaining a sufficient grayscale assigned thereto. 

20 A technique which can be exemplified as a second grayscale 

compression technique relates to an emphasis of high-frequency 
components of an image either before or after the grayscale conversion . 
The second grayscale compression technique is to estimate a portion 
of contrast lost (or supposed to be lost) through the grayscale 

25 conversion, and to compensate the lost portion using a high-frequency 
filter such as for unsharp masking. 

The second grayscale compression technique is advantageous 
in that it does not raise a problem dependent on composition of 
30 image unlike the first grayscale compression technique. The 
high-frequency filter is, however, causative of overshoot at the 
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contour portion of the subject and of noise emphasis at the flat 
portion, and is therefore understood as being not capable of always 
ensuring desirable images. 

5 A technique which can be exemplified as a third grayscale 

compression technique relates to division of a wide-DR image into 
a low-frequency-component image and a high-frequency-component 
image, wherein only the low-frequency-component image is subjected 
to a proper grayscale conversion processing while leaving the 
10 high-frequency-component image unmodified, and the both finally 
added to produce one synthetic image. 

Because the high-frequency-component image is left unmodified 
in the third grayscale conversion technique, lowering in the contrast 

15 due to the grayscale conversion is successfully avoidable . The third 
grayscale conversion technique, however, still suffers from a 
problem of overshoot at the contour portion of the subject, and 
noise emphasis at the flat portion similarly to the second grayscale 
conversion technique, so that there is also proposed a method of 

20 solving the problemby using a non-linear filter (e.g. , median filter) 
in the process of division into the low-frequency-component image 
and high-frequency-component image . 

Summarizing now the first to third grayscale compression 
25 techniques described in the above, they can be classified into those 
effecting the grayscale compression through a relatively local 
processing using neighboring pixels (first and second grayscale 
compression techniques), and that effecting the grayscale 
compression using an entire portion or a relatively large area of 
30 the image (third grayscale compression technique) . The former 
results in an unnatural image having only the high-frequency 
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component thereof enhanced, and is far from successful in obtaining 
effective grayscale compression results. The latter is successful 
in obtaining a more natural image than obtainable by the former 
and is said to be more effective in the grayscale compression, because 
5 it can adjust also components of relatively low frequencies in 
parallel to the emphasis of the high-frequency-components. 

The latter, however, suffers from a problem in that the process 
therefor needs a large-capacity memory mainly for the delay line 

10 or frame memory, so that it is not adaptive to hardware construction. 
For instance, the third grayscale compression technique needs a 
spatial filter for dividing a luminance into a plurality of frequency 
components, wherein it is necessary to incorporate a large amount 
of delay lines into the circuit in order to allow installation of 

15 a large spatial filter, because a non-artificial, effective 
grayscale compression is available only when a large spatial filter 
relative to the image is used. 

Meanwhile, for an exemplary case where a function for 
20 subjecting a wide-DR image to the grayscale compression processing 
is intended to be installed on the output section of an imaging 
apparatus such as digital video camera and digital still camera, 
there is a large need for the function of grayscale compression 
processing of the digital still camera, for example, to be 
25 incorporated into a hardware, because high-speed signal processing 
is necessary in order to output image signals while ensuring a 
predetermined frame rate. Even for a digital still camera for 
photographing still images, for example, there is a demand for 
high-speed grayscale compression processing, because it is necessary 
30 to output a monitored image to a finder in order to determine a 
composition of the image. 
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As described in the above, there is a strong demand for the 
grayscale compression technique which requires only a small memory 
capacity to be consumed and a light load of calculation, allows 
5 easy hardware construction, and ensures a large grayscale 
compression effect. This sort of grayscale compression technique 
has, however, not been proposed yet. 

Additional problems, as described below, commonly reside in 
10 the above -described first to third grayscale compression techniques . 

A first problem relates to generation of overshoot in the 
luminance at the contour portion of the subject in parallel with, 
emphasis of the high-frequency components. 

15 

To suppress this, it is necessary to use a . relatively 
large-sized (20x20 pixels) , two-dimensional, non-linear filter. 
The two-dimensional, non-linear filter of this size expected as 
being realized on the software basis, however, raises a problem 
20 in that cost for the calculation will grow extremely high, and that 
expected as being realized on the hardware basis raises a problem 
in that the circuit scale will grow large due to necessity of a 
large volume of delay lines. 

25 A second problem relates to control of the amount of contrast 

emphasis of high-frequency components in the high-luminance region 
and low-luminance region. The above-described second and third 
grayscale compression techniques are common in that the luminance 
is divided into a low-frequency component and a high-frequency 

30 component, and the grayscale compression is effected by enhancing 
the high-frequency component while keeping the low-frequency 
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component relatively suppressed. 

The emphasis of the high-frequency component , however, results 
in clipping of the luminance at around the maximum luminance and 
5 minimum luminance acceptable by a display apparatus or the like, 
and consequently in loss of detail of the image, so that the grayscale 
conversion could not be said as being appropriate, and this raises 
a need for some countermeasure by which the clipping of luminance 
is avoidable. 

10 

Another problem resides in that an excessive emphasis of the 
contrast results in an image having an unnaturally enhanced contour 
portion of the subject, even under a condition not causative of 
clipping of the luminance. 

15 

DISCLOSURE OF THE INVENTION 

The present invention is conceived in view of the 
aforementioned situation, and an object thereof is to realize a 
20 grayscale compression technique which requires only a small memory 
capacity to be consumed and a less load of calculation, allows easy 
hardware construction, and ensures a large grayscale compression 
effect . 

25 Another ob j ect is to make it possible to appropriately enhance 

the contrast of image using a smaller capacity of memory, based 
on a less amount of calculation, and based on a simple hardware 
construction . 

30 An image processing apparatus of the present invention is 

characterized by comprising a reduced image generation means for 



8 



generating a reduced image from an input image; a correction 
information acquisitionmeans for acquiring a correction information 
of the input image based on the reduced image; and a grayscale 
conversion means for converting grayscale of the input image; wherein 
5 the grayscale conversion means corrects contrast of the input image 
using the correction information, as a processing to be performed 
before and/or after the grayscale is converted. 

The image processing apparatus can further include a smoothing 
10 means for generating a smoothed image having luminance Lc of pixels 
composing the input image smoothed based on interpolation 
calculation using pixels composing the reduced image, wherein the 
grayscale conversion means can be configured so as to generate a 
contrast-corrected image based on luminance Lc of pixels composing 
15 the image, luminance Li of pixels composing the smoothed image, and 
a predetermined gain value g. 



The grayscale conversion means can be configured so as to 
calculate luminance Lu of pixels composing the contrast-corrected 
20 image according to the equation below: 

Lu=g- (Lc-Li) +Li 



The reduction means can be configured so as to divide the input 
image into a plurality of blocks, to calculate an average value 
25 of luminance of pixels which belong to the individual blocks, and 
to produce a reduced image constructed from pixels in the samenumber 
with that of the blocks and having the average value as luminance 
of the pixels. 
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The smoothing means can be configured so as to pinpoint a 
position on the reduced image corresponded to an interpolated 
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position which is a position of a pixel to be interpolated/ and 
to use pixels which reside in the vicinity of the pinpointed position^ 
to thereby calculate luminance Li of pixels of the smoothed image. 

5 The smoothing means can also be configured so as to pinpoint 

a position on the reduced image corresponded to a position of 
interpolation which is a position of a pixel to be interpolated, 
and to use 4x4 pixels which reside in the vicinity of the pinpointed 
position, to thereby calculate luminance Li of pixels of the smoothed 
10 image based on bicubic interpolation. 

The image processing apparatus of the present invention can 
further include a logarithmic conversion means for subjecting 
luminance Lc of pixels composing the image before input to the 
15 smoothing means to logarithmic conversion, and a logarithmic 
inversion means for subjecting luminance of pixels composing the 
contrast-corrected image . 

The image processing apparatus of the present invention can 
20 further include a smoothing means for generating a smoothed image 
having luminance Lc of pixels composing the input image smoothed 
based on interpolation calculation using pixels composing the 
reduced image, and a gain value setting means for setting a gain 
value g used for correcting the contrast; wherein the grayscale 
25 conversion means can be configured so as to generate a 
contrast-corrected image based on luminance Lc of pixels composing 
the input image, luminance Li of pixels composing the smoothed image, 
and a predetermined gain value g; and the gain value setting means 
can be configured so as to set the gain value g based on input initial 
30 gain value go, reference gain value 1, and an attenuation value 
attn(Thi, Th2, Lc) calculated using a first luminance threshold value 
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Thi, a second luminance threshold value Th2, and luminance Lc of 
pixels composing the input image. 

The image processing apparatus of the present invention can 
further include a conversion means for generating a tone-converted 
image by converting luminance L of pixels composing the input image 
based on a conversion function; a smoothing means for generating 
a smoothed image by smoothing luminance Lc of pixels composing the 
tone-converted image; and a gain value setting means for setting 
a gain value g used for correcting the contrast based on an initial 

gain value go which expresses an inverse l/y of a slope y of the 
conversion function; wherein the contrast correction means can be 
configured so as to generate a contrast-corrected image based on 
luminance Lc of pixels composing the tone-converted image, luminance 
Li of pixels composing the smoothed image, and a gain value g; and 
the gain value setting means can be configured so as to set the 
gain value g based on input initial gain value go, reference gain 
value 1, and an attenuation value attn(Thi, Th2, Lc) calculated using 
a first luminance threshold value Thi, a second luminance threshold 
value Th2/ and luminance Lc of pixels composing the tone-converted 
image . 

The gain value setting means can be configured so as to set 
the gain value g according to the equation below: 

25 g=l+ (go-l) -attn (Thi, Th2, Lc) 

The gain value setting means can be configured also so as to 
calculate the attenuation value attn (Thi, Th2, Lc) according to the 
equation below: 
30 attn (Thi, Th2, Lc) = I (Lc-ThJ / (Th2-Thi) | 

(2Thi-Th2^Lc^Th2) 
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attn (Thi, Th2, Lc) =1 

( Lc<2Thi-Th2 , Th2<Lc) 

The grayscale conversion means can be configured so as to 
5 calculate the luminance Lu of pixels composing the contrast-corrected 
image according to the equation below: 

Lu=g- (Lc-Li) +Li 

The first luminance threshold value Thi can be defined as an 
10 intermediate gray level, and the second luminance threshold value 
Th2 can be defined as a maximum white level. 

The reduced image generation means can be configured so as 
to generate a reduced image by converting the input image into the 

15 tone-converted image based on the conversion function and then by 
reducing a size of the tone-converted image, and the correction 
information acquisition means can be configured so as to acquire 
correction information including slope of the conversion function, 
and the grayscale conversion means can be configured so as to correct 

2 0 contrast of the tone-converted image based on the reduced image 
and the slope of the conversion function. 

The image processing apparatus of the present invention can 
further includes a hold means for holding the reduced image 
25 corresponded to a previous frame ' s image and a slope of the conversion 
function applied to the previous frame's image. 

The reduced image generation means can be configured so as 
to convert a pixel value of an image of the current frame by step-wisely 
30 using one or more conversion functions, and the grayscale conversion 
means can be configured so as to generate the contrast-corrected 
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image by correcting contrast of the tone-corrected image based on 
the reduced image held by the hold means and a product of the slopes 
individually corresponded to one or more conversion functions. 

Of one or more conversion functions^ at least one conversion 
function can be configured as a monotonously-increasing function. 

The image processing apparatus of the present invention can 
further include an average value calculation means for calculating 
an average value of pixel values of the tone-converted image, and 
of one or more conversion functions, at least one conversion function 
can be configured so as to have a slope in proportion to an inverse 
of the average value calculated by the average value calculation 
means . 

r 

The average value calculation means can be configured so as 
to divide the tone-corrected image into a plurality of blocks, and 
so as to calculate, as the average value, a value by weighted addition 
of averages of pixel values of the individual blocks. 

The reduced image generation means can be configured so as 
to generate a first reduced image by reducing a size of the 
tone-converted image, and to multiply the individual pixel values 
of the first reduced image by a value in proportion to an inverse 
of an average value of pixel values of the first reduced image, 
to thereby generate a second reduced image. 

The image processing apparatus of the present invention can 
further include a logarithmic conversion means for subjecting pixel 
values of an image in the current frame to logarithmic conversion, 
and a logarithmic inversion means for subjecting pixel values of 
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the contrast-corrected image to the logarithmic inversion. 

The image processing apparatus of the present invention can 
further include a gamma conversion means for subjecting pixel values 
5 of the contrast-corrected image to gamma conversion; a luminance 
range information calculation means for calculating luminance range 
information which indicates a distribution range of luminance 
components of the contrast-corrected image after gone through the 
gamma conversion by the gamma conversion means; and a normalization 
10 means for normalizing, into a predetermined range, the distribution 
of pixel values of the contrast-corrected image after gone through 
the gamma conversion by the gamma conversion means based on the 
luminance range information calculated by the luminance range 
information calculation means. 

15 

The luminance range calculation means can be configured so 
as to calculate, as the luminance range information, an upper limit 
value and a lower limit value of the luminance components of the 
contrast-corrected image after gone through the gamma conversion 

2 0 by the gamma conversion means, and the normalization means can be 
configured so as to convert pixel values of the contrast-corrected 
image so that the upper limit value and the lower limit value of 
the luminance components of the contrast-corrected image calculated 
by the luminance range information calculation means respectively 

25 coincide with the "upper limit value and the lower limit value of 
a range of luminous component reproducible by an assumed reproduction 
apparatus . 

The hold means can be configured so as to hold previous frame ' s 
30 luminance range information calculated by the luminance range 
information calculation means. 
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The image can be a monochrome image constructed from pixels 
having luminance components. 

5 The image can be a color image constructed from pixels having 

a plurality of color components. 

The reduced image generation means can be configured so as 
to generate a first luminance image constructed from pixels having 
10 luminance components based on the color image^ to convert the first 
luminance image into the tone-converted luminance image, and to 
generate a color tone-converted image constructed from pixels having 
a plurality of color components based on the tone-converted luminance 
image . 

15 

The reduced image generation means can be configured so as 
to calculate the individual color components of the tone-converted 
image, by calculating difference values between values of the 
individual color components of the color image and values of the 
20 luminance components, thenby calculating a product of the difference 
values and slope of the conversion function, and by adding the product 
to the values of the individual color components of the tone-converted 
luminance image. 

25 The reduced image generation means can be configured so as 

to calculate the individual color components of the tone-converted 
image, by calculating an average value of the luminance component 
of the first luminance image, then by calculating a coefficient 
in proportion to an inverse of the average value, and by multiplying 

30 values of the individual color components of the color image by 
the coefficient. 
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The grayscale conversion means can be configured so as to 
generate a color contrast-corrected image, by generating a second 
luminance image constructed from pixels having luminance components 
5 based on the color tone-converted image, then by correcting contrast 
of the color tone-converted image generated by the conversion means 
based on the second luminance image, the reduced image held by the 
hold means and the slope of the conversion function, to thereby 
generate the color contrast-corrected image. 

10 

The image processing apparatus of the present invention can 
further include a gamma conversion means for subjecting pixel values 
of the color contrast-corrected image to gamma conversion; a 
luminance range information calculation means for generating a third 

15 luminance image constructed from pixels having luminance component 
based on the color contrast-corrected image after gone through the 
gamma conversion by the gamma conversion means, and for calculating 
luminance range information which indicates a distribution range 
of luminance components of the third luminance image; and a 

20 normalization means for normalizing, into a predetermined range, 
the distribution of pixel values of the color contrast-corrected 
image after gone through the gamma conversion by the gamma conversion 
means based on the luminance range information calculated by the 
luminance range information calculation means. 

25 

The image processing method of the present invention is 
constructed from a reduced image generation step for generating 
a reduced image from an input image; a correction information 
acquisition step for acquiring a correction information of the input 
30 image based on the reduced image; and a grayscale conversion step 
for converting grayscale of the input image; wherein the grayscale 
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conversion step corrects contrast of the input image using the 
correction information, as a processing to be performed before and/or 
after the grayscale is converted. 

5 According to the image processing apparatus and method of the 

present invention, it is made possible to generate a reduced image 
from an input image, to acquire correction information based on 
the generated reduced image, and to convert grayscale of the input 
image. In the processing of the grayscale conversion, contrast of 
10 the input image is corrected using the correction information, as 
a processing to be performed before and/or after the grayscale is 
converted. 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 

Fig. 1 is a block diagram showing an exemplary configuration 
of a digital video camera according to one embodiment of the present 
invention; 

Fig. 2 is a block diagram showing a first exemplary 
20 configuration of a DSP shown in Fig. 1; 

Fig. 3 is a block diagram showing a first exemplary 
configuration of a tone curve correction section shown in Fig. 2; 
Fig. 4 is a graph showing an exemplary tone curve; 
Fig. 5 is a block diagram showing a second exemplary 
25 configuration of the tone curve correction section shown in Fig. 
2; 

Fig. 6 is a block diagram showing a third exemplary 
configuration of the tone curve correction section shown in Fig. 
2; 

30 Fig. 7 is a block diagram showing an exemplary configuration 

of a reduced image generation section shown in Fig. 2; 
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Fig. 8 is a block diagram showing an exemplary configuration 
of an average value calculation section shown in Fig. 7 

Fig. 9 is a block diagram showing an exemplary configuration 
of a contrast correction section shown in Fig. 2; 
5 Fig. 10 is a block diagram showing an exemplary configuration 

of an interpolation section shown in Fig. 9 

Fig. 11 is a drawing for explaining processing of the 
interpolation section shown in Fig. 9; 

Fig. 12 is a block diagram showing an exemplary configuration 
10 of a gain value setting section shown in Fig. 9; 

Fig. 13 is a block diagram showing an exemplary configuration 
of a contrast emphasizing section shown in Fig. 9; 

Fig. 14 is a drawing for explaining processing in a luminance 
range normalization section shown in Fig. 2; 
15 Fig. 15 is a block diagram showing an exemplary configuration 

of a luminance range information calculation section shown in Fig. 
2; 

Fig. 16 is a block diagram showing an exemplary configuration 
of a luminance range normalization section shown in Fig. 2; 
20 Fig. 17 is a block diagram showing an exemplary configuration 

of a composite section substitutable to a portion ranging from the 
tone curve correction section to the contrast correction section 
shown in Fig. 2; 

Fig. 18 is a flow chart for explaining grayscale compression 
25 processing by the first exemplary configuration of the DSP; 

Fig. 19 is a flow chart for explaining details of the processing 
in step SI shown in Fig. 18; 

Fig. 20 is a flow chart for explaining details of the processing 
in step S2 shown in Fig. 18; 
30 Fig. 21 is a block diagram showing a second exemplary 

configuration of the DSP shown in Fig. 1; 
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Fig. 22 is a block diagram showing a first exemplary 
configuration of a tone curve correction section shown in Fig. 21; 

Fig. 23 is a block diagram showing a second exemplary 
configuration of the tone curve correction section shown in Fig. 
5 21; 

Fig. 24 is a block diagram showing a third exemplary 
configuration of the tone curve correction section shown in Fig. 
21; 

Fig. 25 is a block diagram showing an exemplary configuration 
10 of a reduced image generation section shown in Fig. 21; 

Fig. 26 is a block diagram showing an exemplary configuration 
of a contrast correction section shown in Fig. 21; 

Fig. 27 is a block diagram showing an exemplary configuration 
of a composite section substitutable to a portion ranging from the 
15 tone curve correction section to the contrast correction section 
shown in Fig. 21; 

Fig. 28 is a block diagram showing an exemplary configuration 
of a luminance range information calculation section shown in Fig. 
21; 

20 Fig. 29 is a flow chart for explaining grayscale compression 

processing by the second exemplary configuration of the DSP; 

Fig. 30 is a flow chart for explaining details of the processing 
in step S43 shown in Fig. 29; 

Fig . 31 is a flow chart for explaining details of the processing 
25 in step S44 shown in Fig. 29; 

Fig. 32 is a block diagram showing an exemplary configuration 
of an image processing system applied with the present invention; 

Fig. 33 is a flow chart for explaining operation of an image 
processing system shown in Fig. 32; 
30 Fig. 34 is a block diagram showing a first exemplary 

configuration of the image processing apparatus shown in Fig. 32; 



Fig. 35 is a block diagram showing an exemplary configuration 
of a tone curve correction section shown in Fig. 34; 

Fig. 36 is a drawing showing an exemplary tone curve used in 
the first exemplary configuration of the image processing apparatus; 

Fig. 37 is a block diagram showing an exemplary configuration 
of a smoothed luminance generation section shown in Fig. 34; 

Fig. 38 is a block diagram showing an exemplary configuration 
of a reduced image generation section shown in Fig. 37; 

Fig. 39 is a block diagram showing an exemplary configuration 
of an average value calculation section shown in Fig. 38; 

Fig. 40 is a block diagram showing an exemplary configuration 
of an interpolation section shown in Fig. 37; 

Fig. 41 is a block diagram showing an exemplary configuration 
of a gain value setting section shown in Fig. 34; 

Fig. 42 is a block diagram showing an exemplary configuration 
of a contrast correction section shown in Fig. 34; 

Fig. 43 is a flow chart for explaining grayscale-compressed 
image generation processing by the first exemplary configuration 
of the image processing apparatus; 

Fig. 44 is a block diagram showing a second exemplary 
configuration of the image processing apparatus shown in Fig. 32; 

Fig. 45 is a flow chart for explaining grayscale-compressed 
image generation processing by the second exemplary configuration 
of the image processing apparatus; and 

Fig. 4 6 is a block diagram showing an exemplary configuration 
of a general-purpose personal computer. 

BEST MODES FOR CARRYING OUT THE INVENTION 

A digital video camera as one embodiment of the present 
invention will be explained below referring to the attached drawings . 
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Fig. 1 shows an exemplary configuration of a digital video 
camera as one embodiment of the present invention. The digital video 
camera 1 takes a picture of a subject, generates a wide-DR image 
5 having a dynamic range of pixel values wider than a general one, 
saves the image into a predetermined storage medium, and outputs 
the wide-DR image after converting it into a narrow-DR image having 
a dynamic range of pixel values narrower than a general one, to 
a built-in display also used as a composition-determining finder 
10 or as an image monitor, or to external apparatuses. 

The digital video camera 1 is roughly constructed from an 
optical system, a signal processing system, a recording system, 
a display system and a control system. 

15 

The optical system is constructed from a lens 2 for condensing 
a photo-image of a subject, a stop 3 for regulating light energy 
of the photo-image, and a CCD image sensor 4 for generating a wide-DR 
image by photo-electric conversion of the condensed photo-image 
20 at a predetermined frame rate. It is to be noted that the description 
below will deal with two cases where the wide-DR image generated 
by the CCD image sensor 4 is a monochrome image constructed from 
one-channel luminance signal, and is a color image constructed from 
multi-channel (e.g., three-channel) luminance signal. 

25 

The signal processing system is constructed from a correlated 
double sampling circuit (CDS) 5 for reducing noise by sampling the 
wide-DR image output from the CCD image sensor 4, an A/D converter 
6 for effecting AD conversion of the wide-DR image removed with 
30 noise by the correlated double sampling circuit 5 to thereby obtain 
a value having a bit width of, for example, 14 to 16 bits or around. 
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and a DSP (Digital Signal Processor) 7 for effecting grayscale 
compression processing of the wide-DR image output by the A/D 
converter 6. 

5 An image signal having a large number of grayscale, just like 

the wide-DR image output from the A/D converter 6 and having a bit 
width of 14 to 16 bits, cannot fully be reproduced by general video 
signals including luminance Y and color difference signals Cr, Cb, 
but the grayscale thereof is compressed by the grayscale compression 
10 processing by the DSP 7 to a range allowing reproduction by the 
general video signals including luminance Y and color difference 
signals Cr, Cb. The DSP 7 will be detailed referring to Fig. 2 and 
succeeding drawings • 

15 The recording system of the digital video camera 1 is 

constructed from a CODEC (Compression/Decompression) 12 which plays 
a part in encoding the wide-DR image or narrow-DR image received 
from the DSP 7 and in recording it into a memory 13, and in reading 
and encoding the code data stored in the memory 13 and supplying 

20 it to the DSP 7, and a memory 13 for storing the encoded wide-DR 
image or narrow-DR image, constructed from a magnetic disk, optical 
disk, magneto-optical disk, semiconductor or the like. 

The display system is constructed from a D/A converter 9 which 
25 takes a part in DA conversion of the narrow-DR image supplied from 
the DSP 7, a video encoder for outputting the analog narrow-DR image 
output from the D/A converter 9 to a display 11 after converting 
it into a general video signals including luminance Y and 
color-difference signals Cr, Cb, and the display 11 typically is 
30 constructed from an LCD (Liquid Crystal Display) and so forth, which 
functions as a finder or video monitor by displaying an image 



22 



corresponded to the video signals. 

The control system is constructed from a timing generator (TG) 

8 for controlling operation timing of the components from the CCD 
5 image sensor 4 to the DSP 1, an input apparatus 15 for accepting 

various operations by the user, and a CPU (Central Processing Unit) 
14 for controlling the entire portion of the digital video camera 
1. 

10 Next, outline of the operations of the digital video camera 

will be explained. An optical image of a subject (incident light) 
comes through the lens 2 and stop 3 to the CCD image sensor 4, undergoes 
photo-electric conversion by the CCD image sensor 4 , and the obtained 
electric signals expressing pixels of the wide-DR image is removed 

15 with noise by the correlated double sampling circuit 5, digitized 
by the A/D converter 6, and supplied to the DSP 7. 

The DSP 7 takes a part in grayscale compression processing 
of the wide-DR image received from the A/D converter 6 to thereby 
20 generate a narrow-DR image, and outputs it to the D/A converter 

9 or the CODEC 12, or to the both. The narrow-DR image supplied 
to the D/A converter 9 is subjected to DA conversion, and is then 
converted into normal video signals by the video encoder 10, and 
the resultant image is displayed on the display 11. On the other 

25 hand, the narrow-DR image supplied to the CODEC 12 is encoded and 
recorded in the memory 13. 

Here is an end of description on the overall operation of the 
digital video camera 1. 

30 

Next, the DSP 7, which is the key for the present invention. 
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will be described. 

Fig . 2 shows a first exemplary configuration of the DSP 7 adapted 
to the wide-DR image which is a monochrome image. The monochrome 
5 wide-DR image input to the DSP 7 is referred to as wide-DR luminance 
image L, hereinafter. Pixel value (i.e., luminance value) of the 
wide-DR luminance image is expressed as L(p) . In this context, p 
is a vector or coordinate expressing pixel position on the image, 
such as p=(x,y) . It is, therefore, determined to use L(p), which 
10 contains both information of the pixel position and luminance value, 
as being discriminated from L which expresses the wide-DR luminance 
image. The same will apply also to other images and the pixel values 
thereof described later. 

15 The DSP 7 is designed so that luminance L(p) of the wide-DR 

luminance image L is input thereto according to the order of raster. 

In the first exemplary configuration of the DSP 1 , a logarithmic 
conversion section 21 subjects the input luminance L(p) to 

20 logarithmic conversion, and outputs the obtained logarithmic 
luminance logL(p) to a tone curve correction section 22. The tone 
curve correction section 22 applies a preliminarily-obtained tone 
curve to the input logarithmic luminance logL(p) and converts it 
in the direction of compressing the grayscale, and outputs the 

25 obtained logarithmic luminance logLc (p) to a reduced image generation 
section 23 and a contrast correction section 25. The tone curve 

correction section 22 outputs a representative valueywhich expresses 
a slope of the applied tone curve to the contrast correction section 

25 . The representative value y which expresses a slope of the applied 

30 tone curve will simply be referred to as representative value y, 
hereinafter. 
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The reduced image generation section 23 generates a reduced 
image logLci based on the logarithmic luminance logLc(p)f 
corresponded to a single frame, received from the tone curve 
5 correction section 22, and makes a reduced image memory 24 to store 
it. 

The contrast correction section 25 corrects the contrast, which 
was weakened by the tone curve correction, of the logarithmic 
10 luminance logLc{p) of the current frame received from the tone curve 

correction section 22, based on the representative value y and the 
previous frame ' s reduced image logLci held in the reduced image memory 
24, and outputs the obtained logarithmic luminance logLu(p) to a 
logarithmic inversion section 26. The logarithmic inversion 
15 section 2 6 subjects the logarithmic luminance logLu(p) having the 
corrected contrast to logarithmic inversion, and outputs the 
obtained luminance Lu(p) expressed by the normal axis to a gamma 
correction section 27. 

20 The gamma correction section 27 subjects the luminance Lu(p) 

received from the logarithmic inversion section 26 to gamma 
correction in consideration of gamma characteristics of a 
reproduction apparatus (e.g., display 11), and then outputs the 
obtained luminance Y(p) after the gamma correction to a luminance 

25 information calculation section 28 and a luminance range 
normalization section 30. The luminance information calculation 
section 28 calculates luminance range information, which indicates 
a luminance distribution, for each luminance Y(p) corresponded to 
a single frame received from the gamma correction section 27, and 

30 allows them to be held by a luminance range information memory 29. 
It is to be noted herein that the luminance range information refers 



25 

to information indicating a distribution range of luminance within 
one frame, by which luminance Yd closest to darkness and luminance 
Yb closest to brightness are calculated as luminance range information 
[Yd, Yb] . 

5 

The luminance range normalization section 30 converts the 
luminance Y (b) of the current frame received from the gamma correction 
section 21 , based on the previous frame ' s luminance range information 
[Yd/ Yb] held by the luminance range information memory 29, so that 
10 the distribution range thereof coincides with a range which can 
be expressed by a reproduction apparatus (e.g., display 11), and 
outputs the obtained luminance Yn(p) to the succeeding steps as pixel 
values of the narrow-DR image. 

15 As has been described in the above, in the process of the 

grayscale compression processing according to the first exemplary 
configuration of the DSP 7, the reduced image logLci is generated 
by the reduced image generation section 23, and the luminance range 
information [Yd, Yb] is calculated by the luminance range information 

20 calculation section 28 . The reduced image logLci and luminance range 
information [Yd, Yb] are referred to as an intermediate information, 
hereinafter . 

With DSP 7, the intermediate information is calculated for 
25 the individual frames of the input wide-DR image, and the calculated 
intermediate information is used for processing the wide-DR image 
coming one frame after. 

Although it is necessary in general to use information 
30 calculated based on the luminance values for the entire portion 
or a wide range of the image for the purpose of carrying out an 
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effective grayscale compression, a problem arises in the mounting 
in that a time lag before the information is calculated will increase . 
The DSP 7 therefore- uses the previous frame's intermediate 
information to the grayscale compression for the current frame, 
5 by selecting the information extremely less likely to vary with 
time. This configuration makes it possible to avoid expansion of 
the memory consumption and circuit scale even after the mounting. 

Next, details of the first exemplary configuration of the DSP 
10 7 will be described referring to the attached drawings. 

Fig. 3 shows a first exemplary configuration of the tone curve 
correction section 22. In the first exemplary configuration, a LUT 
memory 41 preliminarily holds a lookup table (referred to as LUT, 
15 hereinafter) which corresponds with a monotonously-increasing tone 

curve as shown in Fig . 4 , and a representative value y which expresses 
a slope of the tone curve- It is also allowable that a function 
equivalent to the tone curve may be held in place of the LUT. A 
table reference section 42 corrects the logarithmic luminance 
20 logL(p) into logarithmic luminance logLc(p) based on the LUT held 
in the LUT memory 41. 

Fig. 4 shows an example of the tone curve, wherein the abscissa 
plots input luminance L(p) , and the ordinate plots luminance Lc(p) 

25 after tone curve correction, respectively on the logarithmic axes 
normalized over a range of [0, 1] . Application of the 
monotonously-increasing moderate inverse-S-shaped curve as shown 
in this example will not give so strong effect of grayscale compression 
in the high luminance region and low luminance region, so that it 

30 is possible to obtain a desirable tone with less degree of whiteout 
or blackout even after the grayscale compression. On the contrary. 
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the grayscale compression will strongly affect the middle luminance 
region, but this means that the contrast correction described later 
can fully be applied with the middle luminance region, and results 
in a desirable narrow-DR image with a less degree of contrast 
5 correction also in the middle luminance range. 

It is to be noted that the representative value y expressing 
slope of the tone curve can be determined typically by finding the 
slope values over the entire luminance range and by determining 
10 an average of these values as the representative value y. The tone 
curve shown in Fig. 4 has a representative value y of 0.67. 

Fig. 5 shows a second exemplary configuration of the tone curve 
correction section 22. Unlike the first exemplary configuration, 
15 the second exemplary configuration does not use a 

preliminarily-obtained LUT, but calculates a representative value 

Y for every frame and corrects the logarithmic luminance logL(p) 
into logarithmic luminance logLc(p)- In the second exemplary 
configuration, an average luminance calculation section 51 

20 calculates an average value jJ. of the logarithmic luminance logL(p) 
of one frame. A divider 52 divides a predetermined constant logLT 

by the average value ja, to thereby calculate the representative 
value y. A y memory 53 holds the representative value received from 
the divider 52 . Amultiplier 54 multiplies the logarithmic luminance 
25 logL(p) of the current frame by the previous frame's representative 

value y held by the y memory 53, to thereby calculate the logarithmic 
luminance logLc(p) after the tone curve correction. 

Assuming now that the predetermined constant logLT is defined 
30 as a moderate-level logarithmic luminance, the average value of 
the logarithmic luminance logL(p) for one frame is converted into 
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the logarithmic luminance logLc(p) after the tone curve correction 
having the same value with the logLx. 

Although the representative value y is calculated for every 
5 frame, the value is supposed to differ not largely between the 
preceding frame and succeeding frame, because it is practically 
calculated based on the average value |a of the logarithmic luminance 
logL(p). Hence, also for the representative value y is designed 
to use the one-frame precedence for the tone curve correction for 
10 the current frame, similarly to the above-described reduced image 
logLci and luminance range information [Yd^ Yb] - It is therefore 

defined that also the representative value y is included into the 
intermediate information . 

15 Fig. 6 is a third exemplary configuration of the tone curve 

correction section 22. The third exemplary configuration is, so 
as to say, a combination of the first exemplary configuration and 
the second exemplary configuration. In the third exemplary 
configuration, a LUT memory 61 preliminarily holds a LUT, which 

20 corresponds with the tone curve as shown in Fig. 4, and a 

representative value yi which expresses a slope of the tone curve. 
A table reference section 62 corrects the logarithmic luminance 
logL(p) into logarithmic luminance logLc' (p) based on the LUT held 
in the LUT memory 61k, and outputs it to an average luminance 
25 calculation section 63 and a multiplier 66. 

The average luminance calculation section 63 calculates an 

average value |-i of the logarithmic luminance logLc* (p) for one frame, 
and outputs it to a divider 64 . The divider 64 divides a predetermined 

30 constant logLx by the average value |a, to thereby calculate a 
representative value y2, and allow a y2 memory 65 to store it. A 
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multiplier 66 multiplies the logarithmic luminance logLc' (p) of the 
current frame by the previous frame's representative value 72 held 
by the y memory 65, to thereby calculate the logarithmic luminance 
logLc(p) after the tone curve correction. A multiplier 67 outputs 

5 a product of the representative values Yi, Y2 as a representative 
value Y (=Yi'Y2) to the contrast correction section 25 in the succeeding 
stage . 

Fig. 7 in the next shows an exemplary configuration of the 
10 reduced image generation section 23. A sorting section 71 of the 
reduced image generation section 23 sorts the logarithmic luminance 
logLc (p) for one frame received from the tone curve correction section 
22 in the preceding stage according to blocks to which the luminance 
belongs when the entire image is divided into mxn blocks, and then 
15 supplied to the average value calculating sections 72-1 to 72-N 
(=mxn) . For example, those classified into the first block are 
supplied to average value calculation section 72-1, and those 
classified into the second block are supplied to the average value 
calculation section 72-2 . The same will apply also to the succeeding 
20 ones, and those classified into the N-th block are supplied to the 
average value calculation section 72-N. The following description 
adopts a simple notation of average value calculation section 72 
when there is no need of discrimination of the individual average 
value calculation sections 72-1 to 72-N. 

25 

The average value calculation section 72-i ( i=l , 2 , N) 
calculates an average value of the logarithmic luminance logLc (p) 
classified into the i-th block, out of logarithmic luminance values 
logLc (p) for one frame, and outputs it to a composition section 73. 
30 The composition section 73 generates an m^n-pixel reduced image 
logLci having, as pixel values, the average values of the logarithmic 
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luminance logLc(p) respectively received from the average value 
calculation means 72-i, and makes the reduced image memory 24 in 
the succeeding stage to store it. 

5 Fig. 8 shows an exemplary configuration of the average value 

calculation section 72 . An adder 81 of the average value calculation 
section 72 adds a value held by a register (r) 82 to the logarithmic 
luminance logLc(p) received from the sorting section 71 in the 
preceding stage, to thereby update the value held by the register 
10 (r) 82. A divider 83 divides a value finally held by the register 
82 by the number of pixels Q composing one block, to thereby calculate 
an average value of Q logarithmic luminance logLc (p) values classified 
into one block . 

15 Fig. 9 in the next shows an exemplary configuration of the 

contrast correction section 25. An interpolation position 
designation section 91 acquires a pixel position p of the logarithmic 
luminance logLc(p) received from the tone curve correction section 
22 in the preceding stage (also referred to as position of 

20 interpolation p, hereinafter) , and outputs it to an interpolation 
section 92. The interpolation section 92 calculates, by 
interpolation, the pixel logLci{p) corresponded to the position of 
interpolation p, using the previous frame's reduced image logLci 
held by the reduced image memory 24, and outputs it to. a contrast 

25 enhancement section 94. 

A gain value setting section 93 calculates a gain value g(p) , 
which determines the amount of contrast enhancement of the 
logarithmic luminance logLc(p) of the current frame, based on the 

30 previous frame ' s representative value y received from the tone curve 
correction section 22 and based on the logarithmic luminance logLc (p) 
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of the current frame . The contrast enhancement section 94 calculates 
•the logarithmic luminance logLu(p) having an enhanced contrast in 
the frequency components other than low-frequency ones, based on 
the logarithmic luminance logLc(p) of the current frame, the gain 
5 value g(p) and the interpolation value logLci (p) of the reduced image . 

Fig. 10 shows an exemplary configuration of the interpolation 
section 92. The interpolation section 92 interpolates the pixel 
logLci(p) corresponded to the position of interpolation p based on 
10 bicubic interpolation using 4^4 pixels in the vicinity of the position 
of interpolation p of the previous frame's reduced image logLci- 

A vicinity selection section 101 acquires, upon reception of 
the position of interpolation p, 4x4-pixel pixel value a [4] [4] in 

15 the vicinity of the position of interpolation p, based on the previous 
frame's mxn-pixel reduced image logLci held by the reduced image 
memory 24, and outputs it to the products summation section 104. 
A notation of a[i] [j] herein means that pixel value a is an ixj 
two-dimensional arrangement data. The vicinity selection section 

20 101 outputs horizontal displacement dx and vertical displacement 
dy between the acquired pixel value a [4] [4] and position of 
interpolation p to a horizontal coefficient calculation section 
102 or a vertical coefficient calculation section 103, respectively. 

25 Relationsof the position of interpolationp, neighboring pixel 

value a[4][4] and displacements dx, dy herein will be described 
referring to Fig. 11. 
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An mxn grid shown in Fig. 11 expresses an mxn-pixel reduced 
image logLci. Assuming now that the position of interpolation p= (px, 
py) is given, a position q on the reduced image logLci corresponded 
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to the position of interpolation p will be given as q=(qx, 
qy) = (px/bx-0 . 5, py/by-0 . 5) , where (bx^ by) = (the number of horizontal 
pixel of image logLc/m, the number of vertical pixel of image/n) . 

5 To acquire neighboring pixels around the position q on the 

reduced image corresponded to the position of interpolation p, it 
is recommended to acquire pixels of the reduced image logLci which 
fall within a range of qx-2<x<qx+2 and qy-2<y<qy+2 as indicated 
by hatching in Fig. 11. In the area indicated by the hatching, 4^4 

10 positions marked with + are positions of the pixels to be acquired- 
Displacement (dx, dy) between the neighboring pixel and position 
of interpolation p is defined as a difference with respect to a 
nearest pixel in the left downward direction. That is, displacement 
can be given as (dx, dy)= (decimal portion of qx, decimal portion 

15 of qy) . 

Now referring back to Fig. 10, the horizontal coefficient 
calculation section 102 calculates a horizontal tertiary 
interpolation coefficient kx [4 ] based on the horizontal displacement 
20 dx received from the vicinity selection section 101. Similarly, 
the vertical coefficient calculation section 103 calculates a 
vertical tertiary interpolation coefficient ky[4] based on the 
vertical coefficient based on the vertical displacement dy received 
from the vicinity selection section 101. 

25 

For example, the horizontal tertiary interpolation 
coefficient kx[4] can be calculated using the equation (1) below: 
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z =|dx-i+2 



r (3z -6z^+4)x^6 



k.[i] =i 



(-z^+6z^-12z+8)/6 



0 



(z<1) 
(l<=z<2) 

Otherwise 



...(1) 



Also the vertical tertiary interpolation coefficient ky[4] 
5 can typically be calculated using the equation (2) below: 

z=|dy-j+2| 



k.[j] =i 



(3z^-6z^+4)/'6 (z<l) 
(-z^+6z^-12z-h8)/6 (1<=z<2) 



0 



...(2) 



It is to be noted that any arbitrary calculation formula other 
10 than the equations (1) , (2) shown in the above may be used for the 
calculation of the tertiary interpolation coefficients kx[4] and 
ky[4] so far as a sufficiently smooth interpolation can be obtained. 



The products summation section 104 calculates an interpolation 
15 value Lci(p) of the position of interpolation p of the reduced image 
logLci by sum-of-products calculation using the neighboring pixel 
value a[4][4]^ horizontal interpolation coefficient kx[4] and 
vertical interpolation coefficient ky[4], using the equation (3) 
below : 

logL,,(p) = i i a[i][j]-kx[i]-ky[j] 

20 i=l j=1 

...(3) 
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Next, the gain value setting section 93 will be explained. 
The gain value setting section 93 is, as described in the above, 
used for setting the gain value g{p) which is used for adjusting 
a degree of enhancement of regions other than the low-frequency 
5 region by the contrast enhancement section 94 in the succeeding 
stage. For a gain value of g(p)=l, contrast is not enhanced nor 
suppressed by the contrast enhancement section 94 . For a gain value 
of g(p)>l, contrast is enhanced corresponding to the value. For 
a gain value of g(p)<l, contrast is suppressed corresponding to 
10 the value. 

Setting of the gain value will be described. The contrast 
of the image has already been suppressedby the grayscale compression, 
wherein the amount of suppression depends on slope of the tone curve. 
15 For example, application of the tone curve having a small slope 
in view of effecting a strong grayscale compression means that 
contrast is strongly suppressed. On the other hand, application 
of a straight line having a slope of 1 as the tone curve means that 
the image does not change, or contrast is not suppressed. 

20 

The gain value setting section 93 therefore adopts an inverse 
l/y of the representative value y of the tone curve, for the case 
where the representative value y of the tone curve is smaller than 
1, so that the gain value exceeds 1. 

25 

In another case where the input logarithmic luminance logLc (p) 
is close to the white level or black level, any contrast enhancement 
similar to that applied to the middle luminance region may undesirably 
result in loss of detail of the image due to clipping, therefore 
30 the gain value is adjusted so as to come closer to 1 as the input 
logarithmic luminance logLc(p) comes closer to the white level or 
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black level. 



That is, the gain value g (p) is calculated, assuming an inverse 

of the representative y as l/y=qo, using the equation (4) below: 
g(p)=l+(go-l) xattn(p) 

...(4) 



where, attn(p) is an attenuating coefficient, and is calculated 
by the equation (5) below: 

attn(p) = attnClogLgray. logL^hite^ logLc(P)) 



10 



logL(p)-logL 



gray 



logLwhite-logL 



gray 



( 2 I Og Lgr ay - I Og i te < I Og Lc (P) < I Og j t e ) 



...(5) 



It is to be noted that, in the equation (5) , logLgray represents 
a logarithmic luminance expressing a moderate gray level, and logLwhite 
15 represents a logarithmic luminance expressing the white clipping 
level (maximum white level) , wherein both of which are 
preliminarily-set constants . 



Fig. 12 shows an exemplary configuration of the gain value 
20 setting section 93. A divider 111 calculates the inverse l/Y=go 
of the representative value y received from the preceding stage, 
and outputs it to a subtracter 112. The subtracter 112 calculates 
(go-1) and output it to a multiplier 118. 



25 A subtracter 113 calculates difference ( logLc (p) -logLgray) 

between the logarithmic luminance logLc(p) and the logarithmic 
luminance logLgray having a moderate gray level, and outputs it to 
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a divider 115. The subtracter 114 calculates difference 
(logL„hite~logLgray) between the logarithmic luminance logLwhite having 
a white clipping level and logarithmic luminance logLgray/ nnd outputs 
it to a divider 115. The divider 115 divides the output 
5 (logLc (p) -logLgray) from the subtracter 113 by the output 
(logLwhite-logLgray) f rom the subtracter 114, and outputs the quotient 
to an absolute value calculator 116. The absolute value calculator 
116 calculates an absolute value of the output from the divider 
115, and outputs it to a clipper 117. The clipper 117 clips the 
10 output from the absolute value calculator 116 so as to adjust it 
to 1 when the output exceeds 1, but leaves it unchanged when the 
output does not exceed 1, and outputs the result as attn(p) to a 
multiplier 118. 

15 The multiplier 118 multiplies the output from the subtracter 

112 by the output from the clipper 117, and outputs the product 
to an adder 119 . The adder 119 add 1 to the output from the multiplier 
118, and outputs the result as the gain value g(p) to the succeeding 
stage . 

20 

Fig. 13 in the next shows an exemplary configuration of the 
contrast enhancement section 94. A subtracter 121 calculates 
difference between the logarithmic luminance logLc(p) and the 
interpolation value logLci(p) of the reduced image, and outputs it 

25 to a multiplier 122, The multiplier 122 calculates a product of 
the output from the subtracter 121 and gain value g(p) , and outputs 
it to an adder 123. The adder 123 adds the interpolation value 
logLci(p) of the reduced image to the output of the multiplier 122, 
and outputs thus contrast-corrected logarithmic luminance logLu(p) 

30 to the succeeding stage. 
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It is to be noted now that the interpolation value logLci(p) 
of the reduced image is an interpolated value based on the mxn-pixel 
reduced image, and therefore has only an extremely-low-frequency 
component of the image logLc before being reduced. 

5 

That is, the output ( logLc (p) -logLci (p) ) from the subtracter 
121 is equivalent to that obtained by subtracting only the 
extremely-low-frequency component from the original logarithmic 
luminance logLc (p) . The contrast-corrected logarithmic luminance 

10 logLu(p) is such as being obtained, as described in the above, by 
dividing the luminance signal into two categories of 
extremely-low-frequency component and other components, and of these, 
the components other than the low-frequency components are enhanced 
by being multiplied by the gain value g (p) , and by again synthesizing 

15 the both using the adder 123. 

As is known from the above, the contrast enhancement section 
94 is designed so as to enhance the components ranging from the 
low-to-middle-frequency region to high-frequency region, while 

20 excluding the extremely-low-frequency region, using the same gain 
value g(p) . The contrast-corrected logarithmic luminance logLu(p) 
is therefore free from local overshoot at the edge portion which 
may otherwise be distinct when only the high-frequency region is 
enhanced, and is designed to obtain an image having a contrast enhanced 

25 very naturally to the eyes. 

Next, the luminance range information calculation section 28 
and luminance range normalization section 30 will be explained. 

30 First, outline of the luminance range normalization processing 

will be explained. A purpose of the grayscale compression by the 
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DSP 7 resides in converting a wide-DR luminance image into a narrow-DR 
image adapted to a dynamic range of a reproduction apparatus such 
as the display 11, and for this purpose, a tone curve adapted to 
the dynamic range of the reproduction apparatus is preliminarily 
5 prepared in the tone curve correction section 22. This makes it 
possible to subject most of the photographed wide-DR luminance image 
appropriately to the grayscale compression. 

The dynamic range of incident light may, however, be not 
10 intrinsically so large depending on the subject to be photographed, 
and grayscale compression processing of such image may result in 
an excessive grayscale compression and consequently in confinement 
of the luminance in a range narrower than the dynamic range 
reproducible by the reproduction apparatus. 

15 

To avoid this, the luminance range normalization section 30 
normalizes, as a processing in the final stage of the grayscale 
compression processing, the Iximinance signal Y(p) after the gamma 
correction, so that the dynamic range of the luminance signal Y(p) 
20 after the gamma correction coincides with the dynamic range 
reproducible by the reproduction apparatus. 

Fig. 14 shows a pattern of the luminance range normalization 
processing by the luminance range normalization section 30. In the 
25 line graph of this drawing, the abscissa plots gamma-corrected 
luminance Y before the luminance range normalization, the ordinate 
plots luminance Yn after the luminance range normalization, and the 

grayscale conversion curve a expresses a conversion table used for 
converting the luminance Y into Yn . 

30 

A method of determining the grayscale conversion curve a will 
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be described. The hatched graphic 131 shown under the line graph 
is an exemplary histogram of the luminance image Y before the luminance 
range normalization. In this example, in the stage after the gamma 
correction but before the luminance range normalization, a luminance 
5 image is obtained, of which grayscale being already compressed so 
as to have a dynamic range narrower than a dynamic range ranging 
from a minimum luminance Yinin to a maximum luminance Ymax possibly 
generated by the digital video camera 1. 

10 Because output of the luminance image, while leaving the 

dynamic range thereof unchanged, to the reproduction apparatus 
results in only an inefficient use of the dynamic range reproducible 
by the reproduction apparatus, the normalization is then carried 
out so that the luminance distribution of the luminance image Y 

15 before the luminance range normalization extends over the entire 
portion of the dynamic range of the reproduction apparatus. 

For this purpose, first a range [Yd, Yb] over which the histogram 
graphic 131 of the luminance image Y before the luminance range 

20 normalization distributes, is calculated as a luminance range 
information of the luminance image Y before the luminance range 
normalization. Then luminance values Yna and Yns which fall slightly 
inward from the top and bottom ends of the luminance range [Ynbf 
Ync] of the reproduction apparatus are set, and the grayscale 

2 5 conversion curve a is determined so as to correspond luminances 
{Yniin/ Yd/ Ybf Ymax} on the abscissa with luminance values {Ynbf Yna/ 
Yns/ Ync} on the ordinate. 

Grayscale conversion using this grayscale conversion curve 

30 a is successful in obtaining the luminance image Yn having a histogram 
form such as a hatched graphic 132 shown on the left hand side of 



40 

the line graph. 

A reason why the grayscale conversion curve a is determined 
so that the luminance range [Ya^ Yb] before the luminance range 
5 normalization is mapped to the luminance range [Yna/ Yns] slightly 
narrower than the luminance range [Ynb/Ynd of the reproduction 
apparatus is that sharp luminance clipping at around the luminance 
values Ynb and Ync is prevented from appearing on the image. 

10 It is to be noted herein that the luminance values Yna and 

Yns are preliminarily set with appropriate values based on the 
luminance values Ynb and Ync- 

Acquisition of the luminance range [Ydr Yb] before the luminance 
15 range normalization is carried out by the luminance range information 

calculation section 28, and calculation of the grayscale conversion 

curve a and luminance Yn(p) is executed by the luminance range 
normalization section 30. 

20 Fig . 15 shows an exemplary configuration of the luminance range 

information calculation section 28. In the luminance range 
information calculation section 28 / a decimation section 141 chooses 
the luminance Y(p) received from the gamma correction section 27 
based on the pixel position p. That is, only luminance values of 

25 images at preliminarily-set pixel positions are supplied to a MIN 
sorting section 142 and a MAX sorting section 145 in the succeeding 
stage . 

The MIN sorting section 142 is configured so that k pairs of 
30 a combination of a comparison section 143 and a register 144 are 
arranged in series, and so that the input luminance Y(p) values 
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are held by registers 144-1 to 144-k in an increasing order. 

For example, the comparison section 14 3-1 compares the 
luminance Y(p) from the decimation section 141 and a value in the 
5 register 144-1, and updates, when the luminance Y(p) from the 
decimation section 141 is smaller than the value in the register 
144-1, the value in the register 144-1 using the luminance Y(p) 
from the decimation section 141 • On the contrary, when the luminance 
Y(p) from the decimation section 141 is not smaller than the value 
10 in the register 144-1 , the luminance Y (p) from the decimation section 
141 is supplied to the comparison section 143-2 in the succeeding 
stage . 

The comparison section 143-2 compares the luminance Y (p) from 
15 the comparison section 143-1 and a value in the register 144-2, 
and updates, when the luminance Y(p) from the comparison section 

143- 1 is smaller than the value in the register 144-2, the value 
in the register 144-2 using the luminance Y(p) from the comparison 
section 143-1. On the contrary, when the luminance Y(p) from the 

20 comparison section 143-1 is not smaller than the value in the register 

144- 2, the luminance Y(p) from the comparison section 143-1 is 
supplied to the comparison section 143-3 in the succeeding stage. 

The same will apply also to the comparison sections 143-3 and 
25 thereafter, wherein after completion of input of the luminance Y(p} 
for one frame, the register 144-1 will have the minimum value Yniin 
of the luminance Y(p) held therein, and the registers 144-2 to 144-k 
will have the luminance Y(p) values held therein in an increasing 
order, and the luminance Y(p) held in the register 144-k is output 
30 as the luminance Yd of the luminance range information to the 
succeeding stage. 
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The MAX sorting section 145 is configured so that k pairs of 
a combination of a comparison section 146 and a register 147 are 
arranged in series, and so that the input luminance Y(p) values 
5 are held by registers 147-1 to 147-k in a decreasing order. 

For example, the comparison section 14 6-1 compares the 
luminance Y(p) from the decimation section 141 and a value in the 
register 147-1, and updates, when the luminance Y(p) from the 

10 decimation section 141 is larger than the value in the register 
144-1, the value in the register 147-1 using the luminance Y(p) 
from the decimation section 141. On the contrary, when the luminance 
Y(p) from the decimation section 141 is not larger than the value 
in the register 147-1, the luminance Y (p) from the decimation section 

15 141 is supplied to the comparison section 14 6-2 in the succeeding 
stage . 

The comparison section 146-2 compares the luminance Y(p) from 
the comparison section 146-1 and a value in the register 147-2, 

20 and updates, when the luminance Y(p) from the comparison section 
146-1 is larger than the value in the register 147-2, the value 
in the register 147-2 using the luminance Y(p) from the comparison 
section 146-1. On the contrary, when the luminance Y(p) from the 
comparison section 146-1 is not larger than the value in the register 

25 147-2, the luminance Y(p) from the comparison section 146-1 is 
supplied to the comparison section 146-3 in the succeeding stage. 

The same will apply also to the comparison sections 14 6-3 and 
thereafter, wherein after completion of input of the luminance Y(p) 
30 for one frame, the register 147-1 will have the maximum value Ymax 
of the luminance Y (p) held therein, and the registers 147-2 to 147-k 



will have the luminance Y(p) values held therein in a decreasing 
order, and the luminance Y(p) held in the register 147-k is output 
as the luminance Yb of the luminance range information to the 
succeeding stage. 



Because the luminance Y(p) input to the MIN sorting section 
142 and MAX sorting section 145 are decimated by the decimation 
section 141, appropriate adjustment of intervals of the decimating 
and the number of steps k of the MIN sorting section 142 and MAX 
sorting section 145 makes it possible to obtain luminance values 
Yd, Yb which respectively correspond with 1%, for example, of the 
upper and lower ends of the entire pixels in one frame. 



Fig . 16 shows an exemplary configuration of the luminance range 
normalization section 30. The luminance range normalization 

section 30 determines, as described in the above, the grayscale 

conversion curve a, and converts the gamma-corrected luminance Y (p) 
into the luminance Yn(p) after the luminance range normalization 

using the grayscale conversion curve a. 

Because the grayscale conversion curve a is constructed from 
five line segments as shown in Fig. 14, the luminance range 
normalization section 30 discriminates to which segments the input 
luminance Y(p) belongs, and applies one of five line segments 
composing the grayscale conversion curve a to the input luminance 
Y(p) to thereby convert it into the luminance Yn (p) after the luminance 
range normalization . 

A selector 151 of the luminance range normalization section 
30 outputs, based on the luminance Y(p) input to an input terminal 
i, four luminance values out of the luminance values Ymaxf Yb, Yd, 



44 



Ymin/ Yncf Yns/ Yna and Ynb respectively input to input terminals a 
to h from the output terminals j to m. Correlations thereamong are 
expressed by the equation (6) below: 





^[h, h. c. d] 


i<d 




[h. g. c, d] 


d<i<c 


[j. k, 1, m] = - 


[g. f , b, c] 


c<i<b 




[f . e. a. b] 


b<l<a 




Je, e, a. b] 


a<i 



5 ...(6) 

A subtracter 152 calculates difference between an output from 
the output terminal k and an output from the output terminal j , 
and outputs the result to a divider 155 . A subtracter 153 calculates 

10 difference between an output from the output terminal 1 and an output 
from a subtracter 154, and outputs the result to the divider 155. 
The subtracter 154 calculates difference between the luminance Y(p) 
and an output from the output terminal m, and outputs the result 
to a multiplier 156. The divider 155 calculates ratio of an output 

15 from the subtracter 152 and an output from the subtracter 153, and 
outputs the result to the multiplier 156. The multiplier 156 
calculates product of an output from the divider 155 and an output 
from the subtracter 154, and outputs the result to an adder 157. 
The adder 157 adds an output from the output terminal j and an output 

20 from the multiplier 156, and outputs the result. 

The output Yn (p) from the adder 157 is expressed by the equation 
(7) below, which indicates the line segment of the grayscale 

conversion curve a discriminated based on the gamma-corrected 
25 luminance Y(p). 

Yn(P) = ^(Y(p)-m)+j 
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...(7) 

Here is an end of description on the individual portions 
composing the DSP 7 shown in Fig. 2. 

5 

Meanwhile, it is also possible to reduce the amount of 
calculation by a simpler circuit configuration, paying attention 
to that the average luminance calculation section 63 of the tone 
curve correction section 22 shown in Fig. 6 and the average luminance 

10 calculation section 72 of the reduced image generation section 23 
execute the similar calculation. More specifically, the tone curve 
correction section 22, reduced image generation section 23, reduced 
image memory 24 and contrast correction section 25, all of which 
shown in Fig. 2, can be combined to be provided as a composite section 

15 ais shown in Fig. 17. 

The composite section 160 can be a substitute for the tone 
curve correction section 22, reduced image generation section 23, 
reduced image memory 24 and contrast correction section 25 shown 
20 in Fig. 2. 

A LUT memory 161 of the composite section 160 has a LUT 
corresponded to the tone curve as shown in Fig. 4 and a representative 

value Yi which expresses a slope of the tone curve preliminarily 
25 held therein. A table reference section 162 corrects the logarithmic 
luminance logL(p) received from the preceding stage based on the 
LUT held by the LUT memory 161 to thereby give a logarithmic luminance 
logLc' (p) , and outputs it to a reduced image generation section 163 
and a multiplier 172. 

30 

The reduced image generation section 163 divides the 
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logarithmic luminance image logLc' into mxn blocks, calculates an 
average value of the logarithmic luminance logLc' (p) of the pixels 
which belong to the individual blocks to thereby generate an mxn-pixel 
first reduced image, and makes a first reduced image memory 164 
5 to store it. 



The average luminance calculation section 63 calculates an 

average value \i of pixel values of the previous frame ' s first reduced 
image held by the first reduced image memory 164, and outputs it 
10 to a divider 166. The divider 166 divides a predetermined constant 
logLT by the average value JJ- to thereby calculate the representative 
value Y2, and makes a 72 memory 167 to store it. A multiplier 168 
multiplies the individual pixels of the first reduced image held 
by the first reduced image memory 164 by the representative value 

15 Y2 held by the 72 memory 65 to thereby generate a second reduced image 
logLci, and makes a second reduced image memory 169 to store it. 

A multiplier 170 multiplies the logarithmic luminance 
logLc' (p) of the current frame received from the table reference 
20 section 162 by the previous frame's representative value Y2 held 
by the Y2 memory 167, to thereby calculates the logarithmic luminance 
logLc (p) after the tone curve correction. A multiplier 171 outputs 

a product of the representative values Yi and Y2f as a representative 
value Y (=Yi'Y2) / to a gain value setting section 172. 

25 

The gain value setting section 172 calculates a gain value 
g(p), which determines an amount of contrast enhancement of the 
logarithmic luminance logLc (p) of the current frame, based on the 
representative value Y for the preceding frame received from the 
30 multiplier 171, and the logarithmic luminance logLc (p) for the current 
frame received from the multiplier 170. 
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An interpolation position designation section 173 acquires 
a pixel position p of the logarithmic luminance logLc (p) of the current 
frame received from the multiplier 170 in the preceding stage (also 
5 referred to as position of interpolation p, hereinafter), and outputs 
it to an interpolation section 174. The interpolation section 174 
calculates, by interpolation, the pixel logLci(p) corresponded to 
the position of interpolation p, using the previous frame's second 
reduced image logLci held by the second reduced image memory 169, 
10 and outputs it to a contrast enhancement section 175. 

The contrast enhancement section 175 calculates the 
logarithmic luminance logLu(p) having an enhanced contrast in the 
components other than the low-frequency ones, with respect to the 
15 logarithmic liiminance logLc(p) of the current frame received from 
the multiplier 170, based on the gain value g (p) and the interpolation 
value logLci(p) of the reduced image. 

Use of the composite section 160 allows the average luminance 
20 calculation section 165 to calculate an average value of the mxn-pixel 
first reduced image, and this is successful in reducing the volume 
of calculation as compared with the average luminance calculation 
section 63 shown in Fig. 6 by which the average value is calculated 
for the pixels of the logarithmic luminance image of the original 
25 size. It is therefore possible to reduce the delay time ascribable 
to the calculation. 

Next, a general grayscale compression processing using the 
first exemplary configuration of the DSP 7 applied with the composite 
30 section 160 shown in Fig. 17 will be described referring to the 
flow chart of Fig. 18. 
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In step SI, the DSP 7 converts the input wide-DR luminance 
image L of the current frame into the narrow-DR luminance image 
Yn/ based on the intermediate information (second reduced image 

5 logLc{p), representative value y , luminance range information [Yd^ 
Yb] ) already calculated and held with respect to the previous frame ' s , 
wide-DR luminance image. The DSP 7 also calculates the intermediate 
information with respect to the wide-DR luminance image L of the 
current frame . 

10 

In step S2, the DSP 7 updates the intermediate information 
with respect to the stored previous frame' s wide-DR luminance image, 
using the intermediate information with respect to the calculated 
wide-DR luminance image L of the current frame. 

15 

In step S3, the DSP 7 discriminates whether any succeeding 
frame after the input wide-DR luminance image of the current frame 
is present or not, and upon judgment of the presence, the process 
returns to step SI and processes thereafter are repeated. On the 
. 20 contrary, upon judgment of the absence of any succeeding frame, 
the grayscale compression processing comes to the end. 

Details of the processing on the pixel, basis in step SI will 
be explained referring to the flow chart in Fig. 19. Processing 
25 of the individual steps described below are executed with respect 
to a target pixel (pixel position p) input according to order of 
raster. 

In step Sll, luminance L(p) of a target pixel (pixel position 
30 p) is input to the DSP 7. In step S12, the logarithmic conversion 
section 21 subjects the input luminance L(p) to the logarithmic 
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conversion, and outputs the obtained logarithmic luminance logL(p) 
to the composite section 160. In step S13, the table reference 
section 162 of the composite section 160 corrects the logarithmic 
luminance logL(p) received from the logarithmic conversion section 
5 21 to thereby obtain the logarithmic luminance logLc' (p) based on 
the LUT held by the LUT memory 161, and outputs it to the reduced 
image generation section 163 and multiplier 172. At the same time, 

the LUT memory 161 outputs the representative value Jx of the tone 
curve to the multiplier 171. The multiplier 171 outputs a product 

10 of the representative values Yi and 72 calculated based on the previous 
frame's first reduced image held by the Y2 memory 167, as the 
representative value y, to the gain value setting section 172. 

In step S14, the reduced image generation section 163 generates 
15 the first reduced image based on the logarithmic luminance logLc' (p) 
for one frame after the tone curve correction. Based on the"* first 

reduced image generated herein, the representative value Y2 is 
calculated. The generated first reduced image is also multiplied 

by the calculated representative value Y2/ and thereby the second 
20 reduced image logLci is generated. 

In step S15, the multiplier 170 multiplies the logarithmic 
luminance logLc' (p) of the current frame received from the table 
reference section 162 by the previous frame's representative value 

25 Y2 held by the Y2 memory 167, to thereby calculate the logarithmic 
luminance logLc (p) after the tone curve correction. 

In step S16, the gain value setting section 172 calculates 
the gain value g(p), which determines the amount of contrast 
30 enhancement of the logarithmic luminance logLc (p) of the current 
frame, based on the representative value Y with respect to the 
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preceding frame received from the multiplier 171 and the logarithmic 
luminance logLc(p) of the current frame received from the multiplier 
170. 

5 In step S17, the interpolation section 174 calculates, by 

interpolation, the pixel logLci(p) corresponded to the position of 
interpolation p, using the previous frame's second reduced image 
logLci held by the second reduced image memory 169, and outputs it 
to the contrast enhancement section 175. In step S18, the contrast 

10 enhancement section 175 enhances the components other than 
low-frequency ones of the logarithmic luminance logLc(p) after the 
tone curve correction, based on the interpolation value logLci(p) 
and gain value g(p) of the second reduced image, and outputs the 
obtained contrast-corrected logarithmic luminance logLu{p) to the 

15 logarithmic inversion section 26 in the succeeding stage. 

In step S19, the logarithmic inversion section 26 converts 
the contrast-corrected logarithmic luminance logLu(P) into the 
luminance Lu(P) expressed by the normal axis, and outputs it to the 
20 gamma correction section 27. In step S20, the gamma correction 
section carries out a predetermined gamma correction, and outputs 
the obtained luminance Y(p) to the luminance range information 
calculation section 28 and luminance range normalization section 
30. 

25 

In step S21, the luminance range information calculation 
section 28 generates the luminance range information [Yd, Yb] based 
on the luminance Y{p) for one frame. In step S22, the luminance 
range normalization section 30 normalizes the luminance Y(p) 
30 received from the gamma correction section 27, based on the previous 
frame's luminance range information [Yd, Yb] held by the luminance 
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range information memory 2 9, to thereby calculate the luminance 
Yn(p). In step S23, the luminance range normalization section 30 
outputs the luminance Yn(p) as a pixel value of the 
grayscale-compressed, narrow-DR luminance image. Here is an end 
5 of the detailed explanation of the processing in step SI shown in 
Fig. 18. 

Next, details of the processing in step S2 in Fig. 18 will 
be explained referring to the flow chart in Fig. 20. In step S31, 
10 the reduced image generation section 163 updates the first reduced 
image held by the first reduced image memory 164, using the first 
reduced image generated based on the logarithmic luminance logLc' (p) 
for one frame after the tone curve correction. 

15 In step S32, the divider 166 divides a predetermined constant 

logLx by the average value ]i received from the average luminance 
calculation section 165 to thereby calculate the representative 

value Y2, and updates the representative value 72 held by the 72 memory 
167 using the calculated representative value 72. 

20 

In step S33, the multiplier 168 multiplies the individual 
pixels of the first reduced image updated by the processing in step 
S31 and held by the first reduced image memory 164, by the 

representative value 72 updated by the processing in step S32 and 
25 held by the Y2 memory 65, to thereby generate the second reduced 
image logLci, and updates the second reduced image logLci held by 
the first reduced image memory 169. 

In step S34, the luminance range information calculation 
30 section 28 updates the previous frame's luminance range information 
[Yd/ Yb] held by the luminance range information memory 2 9 using 
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the luminance range information [Yd/ Yb] calculated based on the 
luminance Y(p) for one frame. Here is the end of the detailed 
explanation of the processing of step 32 in Fig. 18. 

5 Next, Fig. 21 shows an exemplary configuration of the DSP 7 

adapted to the wide-DR image, which is a color image. It is to be 
noted that the wide-DR image input to the DSP 7 according to the 
order of raster is not configured so that all of the pixels thereof 
individually have all components of the R, G and B components, but 

10 is configured so as to have either one of the R, G and B components. 
The wide-DR image which is a color image input to the second exemplary 
configuration of the DSP 7 is referred to as a wide-DR color mosaic 
image, hereinafter. Which one of the R, G and B components do the 
individual pixels of the wide-DR color mosaic image have is determined 

15 by the position of the pixels. 

It is defied now to denote a pixel value of the wide-DR color 
mosaic image input to the DSP 7 in the order of raster as L(p) . 

20 In the second exemplary configuration of the DSP 7, a 

demosaicing section 201 demosaics the pixel value L(p) for one frame 
in which each pixel has a different color, so as to make all of 
the pixels have all of R, G and B components, to thereby generate 
a color signal [R (p) , G (p) , B (p) ] , and outputs it to the color balance 

25 adjustment section 202 . An image constructed from the color signals 
output from the demosaicing section 201 will be referred to as a 
wide-DR color image, hereinafter. 

The color balance adjustment section 202 adjusts each of the 
30 R, G and B components so as to make the color balance of the entire 
image appropriate, to thereby generate the color signal [Rb(p) , Gb(p) , 
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Bb(p) ] . It is to be noted that the demosaicing section 201 and color 
balance adjustment section 202 are such as those mounted on general 
digital video recorder equipped with a single-plate-type CCD image 
sensor . 

5 

The logarithmic conversion section 203 subjects the color 
signal [Rb(p)f Gb(p)/ Bb(p)] received from the color balance 
adjustment section 202 to the logarithmic conversion, and outputs 
the obtained logarithmic color signal [logRb (p) / logGb (p) , logBb (p) ] 

10 to the tone curve correction section 204 . The tone curve correction 
section 204 applies a preliminarily-obtained tone curve to the input 
logarithmic color signal [logRb (p) , logGb (p) , logBb(p)] and converts 
it in the direction of compressing the grayscale, and outputs the 
obtained logarithmic color signal [logRb(p), logGb(p)/ logBb(p)] 

15 to a reduced image generation section 205 and a contrast correction 
section 207. The tone curve correction section 204 also outputs 

the representative value y which expresses a slope of the applied 
tone curve to the contrast correction section 207. 

20 The reduced image generation section 205 generates the reduced 

image logLcibasedon the logarithmic color signal [logRc(p) , logGc(p) / 
logBc(p)] for one frame received from the tone curve correction 
section 204, and make the reduced image memory 206 to store it. 

25 The contrast correction section 207 corrects the contrast, 

weakened by the tone curve correction, of the logarithmic color 
signal [logRc(p) , logGc(p) , logBc(p) ] of the current frame received 
from the tone curve correction section 204, based on the 

representative value y and the previous frame's reduced image logLci 
30 held by the reduced image memory 206, and outputs the obtained 
logarithmic color signal [logRu(p), logGu(p), logBu(p)] to the 
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logarithmic inversion section 208. The logarithmic inversion 
section 208 subjects the contrast-corrected logarithmic color signal 
[logRu(p)f logGu(p)/ logBu(p)] to the logarithmic inversion, and 
outputs the obtained color signal [Ru(p)f Gu(p), Bu(p)] expressed 
5 by the normal axis to the gamma correction section 209. 

The gamma correction section 209 subjects the color signal 
[Ru (p) / Gu (p) f Bu (p) ] received from the logarithmic inversion section 
208 to the gamma correction in consideration of the gamma 
characteristic of a reproduction apparatus (e.g., display 11) , and 
outputs the obtained gamma-corrected color signal [Rg(p), Gu(p)/ 
Bg(p)] to the luminance information calculation section 210 and 
luminance range normalization section 212. The luminance 
information calculation section 210 converts the [Rg(p)f Gu(p)/ 
Bg(p)] for one frame, received from the gamma correction section 
209, into the luminance Y(p), calculates the luminance range 
information for indicating the distribution of the luminance Y (p) , 
and makes the luminance range information memory 211 to store it. 
The luminance range information described herein refers to an 
information indicating a range of distribution of the luminance 
Y(p) for one frame, and is typically calculated as the luminance 
range information [Yd, Yb] using the luminance Yd closest to the 
darkness and the luminance Yb closest to the brightness. 

25 The luminance range normalization section 212 converts the 

color signal [Rg(p), Gu(p)/ Bg(p)] of the current frame received 
from the gamma correction section 209 so that the distribution range 
thereof can coincide with a range expressible by the reproduction 
apparatus (e.g., display 11), based on the previous frame ' s luminance 

30 range information [Yd/ Yb] held by the luminance range information 
memory 211, and outputs the obtained color signal [Rn(p)/ Gn(p)/ 
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Bn(p) ] as the narrow-DR image, which is a color image, to the succeeding 
stage • 

As will be described below, the second exemplary configuration 
5 of the DSP 7 adapted to color image is almost similar to the first 
exemplary configuration adapted to monochrome image shown in Fig. 
2 except that the demosaicing section 201 and the color balance 
section 202 are added, but is slightly modified in the internal 
configurations of the individual sections so as to be adapted to 
10 the color image. 

Fig. 22 shows a first exemplary configuration of the tone curve 
correction section 204. In the first exemplary configuration, a 
luminance generation section 221 generates the logarithmic luminance 
15 logLb(p) by calculating a linear sum of the input logarithmic color 
signal [logRb (p) , logGb (p) , logBb (p) ] , and outputs it to the 
subtracters 222-R to 222-B, and to the table reference section 224. 

The subtracter 222-R subtracts the logarithmic luminance 
20 logLb(p) from the logarithmic color signal logRb(p), and outputs 
the result to a multiplier 225-R. A LUT memory 223 has a LUT which 
corresponds to the tone curve as shown in Fig. 4 and a representative 

value Y indicating a slope of the tone curve previously held therein. 
The table reference section 224 corrects the logarithmic luminance 
25 logL(p) into the logarithmic luminance logLc(p) using the LUT held 
by the LUT memory 223, and output it to adders 226-R to 226-B. 

The multiplier 225-R multiplies the output of the subtracter 

222-R by the representative value y received from the LUT memory 
30 223, and outputs it to the adder 226-R. The adder 226-R calculates 
a sum of the output of the multiplier 225-R and the logarithmic 
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luminance logLc (p) , and outputs the results as the logarithmic color 
signal logRc{p) after the tone curve correction to the succeeding 
stage . 

5 It is to be noted now that any constituents for processing 

the G and B components are similar to those for processing the 
above-described R component, so that explanations therefor will 
be omitted. 

10 Fig. 23 shows a second exemplary configuration of the tone 

curve correctipn section 204 . In the second exemplary configuration, 
a luminance generation section 231 calculates a linear sum of the 
input logarithmic color signal [logRb(p), logGb(p)/ logBb(p)] to 
thereby generate the logarithmic luminance logLb(p), and outputs 

15 it to an average luminance calculation section 232. The average 

luminance calculation section 232 calculates an average value |a of 
the logarithmic luminance logL(p) for one frame, and outputs it 
to a divider 233. The divider 233 divides a predetermined constant 
by the average value \x to thereby calculates the representative 
20 value y, and makes a y memory 234 to store it. 

A multiplier 235-R multiplies the logarithmic color signal 
logRb (p) of the current frame by the previous frame ' s representative 

value y held by the y memory 234 , to thereby calculates the logarithmic 
25 color signal logRc(p) after the tone curve correction. 

It is to be noted now that any constituents for processing 
the G and B components are similar to those for processing the 
above-described R component, so that explanations therefor will 
30 be omitted. 
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Fig. 24 shows a third exemplary configuration of the tone curve 
correction section 204. The third exemplary configuration is, so 
as to say, a combination of the first exemplary configuration and 
second exemplary configuration. In the third exemplary 
5 configuration, a luminance generation section 241 calculates a 
linear sum of the logarithmic color signal [logRb(p), logGb(p), 
logBb(p)] to thereby generate the logarithmic luminance logLb(p), 
and outputs it to subtracters 242-R to 242-B, and to a table reference 
section 244. 

10 

The subtracter 242-R subtracts the logarithmic luminance 
logLb(p) from the logarithmic color signal logRb(p), and outputs 
the result to a multiplier 250-R. A LUT memory 243 has a LUT which 
corresponds to the tone curve as shown in Fig . 4 and the representative 

15 value y indicating a slope of the tone curve previously held therein. 
The table reference section 244 corrects the logarithmic luminance 
logL(p) into the logarithmic luminance logLc' (p) using the LUT held 
by the LUT memory 243, and outputs it to an average luminance 
calculation section 245 and to a multiplier 249. 

20 

The average luminance calculation section 245 calculates an 

average value (J. of the logarithmic luminance logLc* (p) for one frame, 
and outputs it to a divider 246. The divider 246 divides a 

predetermined constant logLT by the average value |a to thereby 
25 calculate the representative value Y2, and makes the 72 memory 247 
to store it. The multiplier 248 outputs a product of the 

representative values yi and 72 as the representative value y (=yi* 
y2) to the contrast correction section 207 in the succeeding stage. 

The multiplier 249 multiplies the logarithmic luminance 
30 logLc' (p) of the current frame by the previous frame ' s representative 
value y2 held by the y2 memory 247 to thereby calculate the logarithmic 



58 



luminance logLc(p) after the tone curve correction, and outputs it 
to adders 251-R to 251-B. 

The multiplier 250-R multiplies the output of the subtracter 

5 242-R by the representative value y received from the multiplier 
248, and outputs the result to the adder 251-R. The adder 251-R 
calculates a product of an output of the multiplier 250-R and. an 
output of themultiplier 24 9, and outputs the result as the logarithmic 
color signal logRc(p) after the tone curve correction to the 
10 succeeding stage. 

It is to be noted now that any constituents for processing 
the G and B components are similar to those for processing the 
above-described R component, so that explanations therefor will 
15 be omitted. 

Fig. 25 in the next shows an exemplary configuration of the 
reduced image generation section 205. A luminance generation 
section 261 of the reduced image generation section 205 calculates 
20 a linear sum of the input logarithmic color signal [logRc ip) , logGc (p) , 
logBc(p)] after the tone curve correction to thereby generate the 
logarithmic luminance logLc(p) , and outputs it to a sorting section 
262. 

25 A sorting section 262 sorts the logarithmic luminance logLc (p) 

values according to blocks to which the luminance belongs when the 
entire image is divided into m^n blocks, and then supplies them 
to the average value calculating sections 263-1 to 263-N (=mxn) . 
For example, those classified into the first block are supplied 

30 to the average value calculation section 263-1, and those classified 
into the second block are supplied to the average value calculation 
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section 263-2. The same will apply also to the succeeding ones, 
and those classified into the N-th block are supplied to the average 
value calculation section 263-N. 

5 The average value calculation means 263-i (i=l, 2, N) 

calculates an average value of the logarithmic luminance logLc(p) 
classified into the i-th block, out of logarithmic luminance values 
logLc(p) for one frame, and outputs it to a composition section 264. 
The composition section 2 64 generates an mxn-pixel reduced image 
10 logLci having, as pixel values, the average values of the logarithmic 
luminance logLc(p) respectively received from the average value 
calculation section 263-i, and makes the reduced image memory 206 
in the succeeding stage to store it. 

15 Fig. 2 6 in the next shows an exemplary configuration of the 

contrast correction section 207. The luminance generation section 
270 of the contrast correction section 25 calculates a linear sum 
of the input logarithmic color signal [logRc(p) , logGc(p) , logBc(p) ] 
after the tone curve correction to thereby generate the logarithmic 

20 luminance logLc(p), and outputs it to an interpolation position 
designation section 271 and a gain value setting section 273. 

The interpolation position designation section 271 acquires 
a pixel position p of the logarithmic luminance logLc(p) (also 

25 referred to as position of interpolation p, hereinafter), and outputs 
it to an interpolation section 272. The interpolation section 272 
calculates, by interpolation, the pixel logLci(p) corresponded to 
the position of interpolation p, using the previous frame's second 
reduced image logLci held by the reduced image memory 206, and outputs 

30 it to subtracters 274-R to 274-B, and to adders 276-R to 276-B- 
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The gain value setting section 273 calculates the gain value 
g(p), which determines the amount of contrast enhancement of the 
logarithmic luminance logLc(p) of the current frame, based on the 

representative value y with respect to the preceding frame received 
5 from the tone curve correction section 22 and the logarithmic 

luminance logLc (p) of the current frame, and outputs it to multipliers 
275-R to 275-B. 

The subtracter 274-R subtracts the interpolation value 
10 logLci(p) from the logarithmic color signal logRc(p) and outputs 
the result to the multiplier 27 5-R. The multiplier 275-R multiplies 
the output of the subtracter 274-R by the gain value g (p) , and outputs 
the result to the adder 27 6-R. The adder 27 6-R adds the interpolation 
value logLci(p) to the output of the multiplier 275-R, and outputs 
15 the obtained logarithmic color signal logRu(p) after the contrast 
correction to the succeeding stage. 

It is to be noted now that any constituents for processing 
the G and B components are similar to those for processing the 
20 above-described R component, so that explanations therefor will 
be omitted. 

Fig. 27 in the next shows an exemplary configuration of a 
composite section 300 which can be a substitute for the tone curve 
25 correction section 204, reduced image generation section 205, 
reduced image memory 206 and contrast correction section 207 shown 
in Fig. 21. 

The luminance generation section 301 of the composite section 
30 300 calculates a linear sum of the input logarithmic color signal 
[logRb(p), logGb(p), logBb(p)] to thereby generate the logarithmic 
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luminance logLb(p)f and outputs it to subtracter 302-R to 302-B, 
and to a table reference section 304 . The subtracter 302-R subtracts 
the logarithmic Iximinance logLb (p) from the logarithmic color signal 
logRb(p)/ and outputs the result to a multiplier 316-R. 

5 

A LUT memory 303 of the composite section 300 has a LUT 
corresponded to the tone curve as shown in Fig. 4 and the 

representative value Yi which expresses a slope of the tone curve 
preliminarily held therein. A table reference section 304 corrects 
10 the logarithmic luminance logL(p) received from the luminance 
generation section 301 based on the LUT held by the LUT memory 303 
to thereby give the logarithmic luminance logLc' (p) , and outputs 
it to a multiplier 305 and a reduced image generation section 306. 

15 The multiplier 305 multiplies the logarithmic luminance 

logLc' (p) of the current frame received from the table reference 
section 304 by the previous frame's representative value 72 held 
by a 72 memory 167, to thereby calculate the logarithmic luminance 
logLc(p) after the tone curve correction, and outputs it to adders 

20 317-R to 317-B. 

The reduced image generation section 306 divides the 
logarithmic luminance image logLc' into m^n blocks, calculates 
average values of the logarithmic luminance logLc' (p) values of the 
25 pixels which belong to the individual blocks to thereby generate 
an mxn-pixel first reduced image, and makes a first reduced image 
memory 307 to store it. 



30 



An average luminance calculation section 308 calculates an 

average value M- of pixel values of the previous frame ' s first reduced 
image held by the first reduced image memory 307, and outputs it 
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to a divider 309. The divider 309 divides a predetermined constant 
logLT by the average value to thereby calculate the representative 
value 72/ and makes a Y2 memory 310 to store it. A multiplier 311 
calculates a product of the representative values Yi and Y2 as the 
5 representative value Y (=yi*Y2) / and outputs it to a gain value setting 
section 315 and the multipliers 316-R to 316-B. 

The multiplier 312 multiplies the individual pixels of the 
first reduced image held by the first reduced image memory 164 by 

10 the representative value Y2 heldby the Yan^emory 310 to thereby generate 
the second reduced image logLcif and makes the second reduced image 
memory 313 to store it. 

An interpolation section 314 calculates, by interpolation, 
15 the pixel logLci(p) corresponded to the position of interpolation 
p (also referred to as position of interpolation p, hereinafter) 
of the logarithmic luminance logLc(p) of the current frame received 
from the multiplier 17 0, using the previous frame's second reduced 
image logLci held by the reduced image memory 169, and outputs it 
20 to subtracters 318-R to 318-B, and to adders 320-R to 320-B. 

The gain value setting section 315 calculates the gain value 
g(p), which determines the amount of contrast enhancement of the 
logarithmic luminance logLc(p) of the current frame, based on the 

25 representative value Y with respect to the preceding frame received 
from the multiplier 311 and the logarithmic luminance logLc(p) of 
the current frame received from the multiplier 305, and outputs 
it to multipliers 319-R to 319-B. 

30 The multiplier 316-R calculates a product of an output of the 

subtracter 302-R and the representative value y, and outputs it to 
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the adder 317-R. The adder 317-R calculates a sum of the output 
of the multiplier 316-R and the output of the multiplier 305, and 
outputs it to the subtracter 318-R. The subtracter 318-R subtracts 
the interpolation value logLci (p) from the output of the adder 317-R, 
5 and outputs the result to the multiplier 319-R. The multiplier 319-R 
multiplies the output of the subtracter 318-R by the gain value 
g(p), and outputs the result to the adder 320-R. The adder 320-R 
calculates a sum of the output of the multiplier 319-R and the 
interpolation value logLci(p), and outputs the obtained 
10 contrast-corrected logarithmic color signal logRu(p) to the 
succeeding stage. 

It is to be noted now that any constituents for processing 
the G and B components are similar to those for processing the 
15 above-described R component, so that explanations therefor will 
be omitted. 

Use of the composite section 300 allows the average luminance 
calculation section 308 to calculate an average value of themxn-pixel 

20 first reduced image, and this is successful in reducing the volume 
of calculation as compared with the average luminance calculation 
section 245 shown in Fig. 24 by which the average value is calculated 
for the pixels of the logarithmic luminance image logLc of the original 
size. It is therefore possible to reduce the delay time ascribable 

25 to the calculation. 

Fig. 28 in the next shows an exemplary configuration of the 
luminance range information calculation section 210. In the 
luminance range information calculation section 210, a luminance 
30 generation section 331 calculates a linear sum of the gamma-corrected 
color signal [Rg(p) , Gg(p) , Bg(p) ] to thereby generate the Iximinance 



64 



Y(p), and outputs it to a decimation section 332. The decimation 
section 332 chooses the luminance Y(p) received from the luminance 
generation section 331 based on the pixel position p. That is, only 
luminance values of the pixels at preliminarily-set pixel positions 
5 are supplied to a MIN sorting section 333 and a MAX sorting section 
336 in the succeeding stage. 

The MIN sorting section 333 is configured so that k pairs of 
a combination of a comparison section 334 and a register 335 are 
10 arranged in series, and so that the input luminance Y{p) values 
are held by registers 335-1 to 335-k in an increasing order. 

For example, the comparison section 334-1 compares the 
luminance Y{p) from the decimation section 332 and a value in the 

15 register 335-1, and updates, when the luminance Y(p) from the 
decimation section 332 is smaller than the value in the register 
335-1, the value in the register 335-1 using the li:iminance Y(p) 
from the decimation section 332 . On the contrary, when the luminance 
Y(p) from the decimation section 332 is not smaller than the value 

20 in the register 335-1, the luminance Y (p) from the decimation section 
332 is supplied to the comparison section 334-2 in the succeeding 
stage . 

The same will apply also to the comparison sections 334-2 and 
25 thereafter, wherein after completion of input of the luminance Y(p) 
for one frame, the register 335-1 will have the maximum value Yjuin 
of the luminance Y(p) held therein, and the registers 335-2 to 335-k 
will have the luminance Y(p) values held therein in an increasing 
order, and the luminance Y(p) held in the register 335-k is output 
30 as the luminance Yd of the luminance range information to the 
succeeding stage . 
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The MAX sorting section 336 is configured so that k pairs of 
a combination of a comparison section 337 and a register 338 are 
arranged in series, and so that the input luminance Y(p) values 
5 are held by registers 338-1 to 338-k in a decreasing order. 

For example, the comparison section 337-1 compares the 
luminance Y(p) from the decimation section 332 and a value in the 
register 338-1, and updates, when the luminance Y(p) from the 

10 decimation section 332 is larger than the value in the register 
338-1, the value in the register 338-1 using the luminance Y(p) 
from the decimation section 332 . On the contrary, when the luminance 
Y{p) from the decimation section 332 is not larger than the value 
in the register 338-1, the luminance Y (p) from the decimation section 

15 332 is supplied to the comparison section 337-2 in the succeeding 
stage. 

The same will apply also to the comparison sections 337-2 and 
thereafter, wherein after completion of input of the luminance Y(p) 

20 for one frame, the register 338-1 will have the maximum value Ymax 
of the luminance Y(p) held therein, and the registers 338-2 to 338-k 
will have the luminance Y{p) values held therein in a decreasing 
order, and the luminance Y(p) held in the register 338-k is output 
as the luminance Yb of the luminance range information to the 

25 succeeding .stage. 

Because the luminance Y(p) values input to the MIN sorting 
section 333 and MAX sorting section 336 are decimated by the decimation 
section 332, appropriate adjustment of intervals of the decimating 
30 and the number of steps k of the MIN sorting section 333 and MAX 
sorting section 336 makes it possible to obtain luminance Yd, Yb 
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values which correspond with 1% or 0.1%, for example, of the upper 
and lower ends of the whole pixels in one frame. 

Next, a general grayscale compression processing using the 
5 second exemplary configuration of the DSP 7 applied with the composite 
section 300 shown in Fig. 27 will be described referring to the 
flow chart of Fig. 29. 

In step S41, the DSP 7 (demosaicing section 201) demosaics 
10 a wide-DR color mosaic image to thereby generate a wide-DR color 
image, and outputs the pixel value thereof, that is, a color signal 
[R(p), G(p), B(p)] to the color balance adjustment section 202 in 
the order of raster. In step S42, the DSP 7 (color balance adjustment 
section 202) respectively adjust the R, G and B components so that 
15 a color balance of the entire image will become appropriate, to 
thereby generate the color signal [Rb(p), Gb(p), Bb(p)]. 

In step S43, the DSP 7 converts the color signal of the input 
wide-DR color image L of the current frame into the narrow-DR color 
20 image Yn, based on the intermediate information ( second reduced image 

logLc(p), representative value y, and luminance range information 
[Yd, Yb] ) already calculated and held with respect to the previous 
frame's, wide-DR color image. The DSP 7 also calculates the 
intermediate information with respect to the wide-DR color image 
25 L of the current frame. 

In step S4 4, the DSP 7 updates the intermediate information 
with respect to the stored previous frame's wide-DR color image, 
using the intermediate information with respect to the calculated 
30 wide-DR color image L of the current frame. 
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In step S45, the DSP 7 discriminates whether any succeeding 
frame after the input wide-DR color image of the current frame is 
present or not, and upon judgment of the presence, the process returns 
to step S41 and processes thereafter are repeated. On the contrary, 
5 upon judgment of the absence of any succeeding frame, the grayscale 
compression processing comes to the end. 

Details of the processing on the pixel basis in step S42 will 
be explained referring to the flow chart in Fig. 30. Processing 
10 of the individual steps described below are executed with respect 
to a target pixel (pixel position p) input according to order of 
raster. 

In step S51, the color balance adjustment section 202 outputs 
15 the generated color signal [Rb(p) ^ Gb(p) , Bb(p) ] to the logarithmic 
conversion section 203. In step S52, the logarithmic conversion 
section 203 subjects the color signal [Rb(p)/ Gb(p)f Bb(p)] to the 
logarithmic conversion, and outputs the obtained logarithmic color 
signal [logRb(p) , logGb(p) , logBb(p) ] to the composite section 300. 

20 

In step S53, the luminance generation section 301 of the 
composite section 300 calculates a linear sum of the input logarithmic 
color signal [logRb(p) , logGbCp) , logBb(p) ] to thereby generate the 
logarithmic luminance logLb(p)f and outputs it to the subtracters 
25 302-R to 302-B, and to the table reference section 304. In step 
S54, table reference section 304 corrects the input logarithmic 
luminance logL(p) into the logarithmic luminance logLc' (p) based 
on the LUT held by the LUT memory 303, and outputs it to the multiplier 
305 and reduced image generation section 306. 

30 

In step S55, the reduced image generation section 306 generates 
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the first reduced image based on the logarithmic luminance logLc' (p) 
for one frame after the tone curve correction. Herein the 

representative value 72 is calculated based on the generated first 
reduced image. The second reduced image logLci is also generated 
5 herein by multiplying the generated first reduced image by thus 
calculated representative value 72. 

In step S56, the multiplier 305 multiplies the logarithmic 
luminance logLc' (p) of the present image received from the table 
10 reference section 304 by the previous frame's representative value 

Y2 held in the Y2 memory 310 , to thereby calculate the logarithmic 
luminance logLc(p) after the tone curve correction. 

In step S57, calculations are carried out for the R component 
15 by the subtracter 302-R, multiplier 316-Rand adder 317-R, to generate 
the logarithmic color signal logRc (p) after the tone curve correction . 
For G component, calculations are carried out by the subtracter 
302-G, multiplier 316-G and adder 317-G, to generate the logarithmic 
color signal logGc(p) after the tone curve correction. For B 
20 component, calculations are carried out by the subtracter 302-B, 
multiplier 316-B and adder 317-B, to generate the logarithmic color 
signal logBc(p) after the tone curve correction. 

In step S58, the gain value setting section 315 calculates 
25 the gain value g{p)r which determines the amount of contrast 
enhancement of the logarithmic luminance logLc(p) of the current 

frame, based on the representative value y with respect to the 
preceding frame received from the multiplier 311 and the logarithmic 
luminance logLc(p) of the current frame received from the multiplier 
30 305. In step S59, the interpolation section 314 calculates, by 
interpolation, the pixel logLci(p) corresponded to the position of 
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interpolation using the previous frame's second reduced image 
logLci held by the second reduced image memory 313. 

In step S60/ calculations are carried out for the R component 
5 by the subtracter 318-R, multiplier 319-R and adder 320-R, to generate 
the logarithmic color signal logRu (p) after the tone curve correction. 
For G component, calculations are carried out by the subtracter 
318-G, multiplier 319-G and adder 320-G, to generate the logarithmic 
color signal logGu(p) after the tone curve correction. For B 
10 comppnent, calculations are carried out by the subtracter 318-B, 
multiplier 319-B and adder 320-B, to generate the logarithmic color 
signal logBu(p) after the tone curve correction. 

In step S61, the logarithmic inversion section 208 subjects 
15 the logarithmic color signal [logRu(p), logGu(p)/ logBu(p)] after 
the contrast correction to the logarithmic inversion to thereby 
generate the color signal tRu(p)f Gu(p)f Bu(p)] expressed by the 
normal axis, and outputs it to the gamma correction section 209. 
In step S62, the gamma correction section 209 carries out a 
20 predetermined gamma correction,, and outputs the obtained 
gamma-corrected color signal [Rg(p) , Gg(p) , Bg(p) ] to the luminance 
information calculation section 210 and luminance range 
normalization section 212. 

25 In step S63, the luminance generation section 331 of the 

luminance range information calculation section 210 generates the 
luminance Y(p) based on the gamma-corrected color signal [Rg(p), 
Gg(p), Bg(p)]. In step S64, the MIN sorting section 333 and MAX 
sorting section 336 of the luminance range information calculation 

30 section 210 calculate the luminance range information [Yd, Yb] based 
on the luminance Y{p) for one frame. 
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In step S65, the luminance range normalization section 212 
normalizes the color signal [Rg(p) , Gg(p) , Bg(p) ] received from the 
gamma correction section 209 based on the previous frame * s luminance 
5 range information [Y<i, Yb] held by the luminance range information 
memory 211^ to thereby calculate the color signal [Rn(p)/ Gn(p), 
Bn(p)]. In step S66, luminance range normalization section 212 
outputs thus calculated color signal [Rn(p) , Gn(p) , Bn(p) ] as a pixel 
value of the grayscale-compressed, narrow-DR color image. Here is 
10 the end of the detailed explanation of the processing of step S43 
in Fig. 29. 

Next, details of the processing in step S44 in Fig. 29 will 
be explained referring to the flow chart in Fig. 31. In step S71, 
15 the reduced image generation section 306 updates the first reduced 
image held by the first reduced image memory 307, using the first 
reduced image generated based on the logarithmic luminance logLc' (p) 
for one frame after the tone curve correction. 

20 In step S72, the divider 309 divides a predetermined constant 

logLx by the average value \x received from the average luminance 
calculation section 165 to thereby calculate the representative 

value 72, and updates the representative value Y2 held by the Y2 memory 
310 using thus calculated representative value Y2. 

25 

In step S73, the multiplier 312 multiplies the individual 
pixels of the first reduced image held by the first reduced image 
memory 307 updated by the processing of step S71 by the representative 

value Y2 held by the Y2 memory 310 updated by the processing of step 
30 S72, to thereby generate the second reduced image logLcif and updates 
the second reduced image logLci held by the first reduced image memory 
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313. 

In step S74, the luminance range information calculation 
section 210 updates the previous frame ' s luminance range information 
5 [Ydf Yb] held by the luminance range information memory 211, using 
the luminance range information [Yd, Yb] generated based on the [Rg(p) , 
Gg{p), Bg(p)] for one frame. Here is the end of the detailed 
explanation of the processing of step S44 in Fig. 29. 

10 Here is the end of the detailed explanation of the second 

exemplary configuration of the DSP 7. 

It is tobe noted, for example, that each of the average luminance 
calculation section 51 shown in Fig . 5, average luminance calculation 

15 section 63 shown in Fig. 6, average luminance calculation section 
165 shown in fig. 17, average luminance calculation section 232 
shown in Fig. 23 and average luminance calculation section 245 shown 
in Fig. 24 was configured so that an average value of the luminance 
value was calculated, wherein the calculation for finding the average 

20 value may adopt weighted average . For example, a larger weight given 
to the central portion of an image rather than to the peripheral 
portion enables brightness correction while placing a stress on 
reflectance of a subject which resides in the central portion of 
the image. 

25 

The composite section 160 shown in Fig. 17 and the composite 
section 300 shown in Fig. 27 have the memory for holding the generated 
first reduced image and the memory for holding the second reduced 
image generated by multiplying thus generated first reduced image 

30 by the representative value Y2, wherein these two memories may be 
combined into one because the first reduced image is no more necessary 
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to be held once the second reduced image is generated. 

If the present invention is applied to a digital video camera 
which takes a picture of a wide-DR image, compresses the grayscale 
5 and outputs it as an image displayable on a display having a narrow 
dynamic-range as in the present embodiment, it is made possible 
to realize the grayscale compression processing by a configuration 
having only a largely reduced capacity of memory (used for frame 
memory or delay line of the pixel series data) , which has otherwise 
10 been indispensable for the conventional grayscale compression 
technique, and it is also made possible to obtain an output image 
which is by no means inferior to that obtained by the grayscale 
compression processing conventionally been realized using a 
large-scale filtering. 

15 

This makes it possible to realize a high-quality and 
inexpensive digital video camera which has never been realized. 

The wide-DR image in the present embodiment was subjected to 
20 the grayscale compression processing assuming the display 11 as 
a reproduction apparatus, wherein it is also possible, for example,, 
to carry out the grayscale compression processing so as to be adapted 
to dynamic ranges expressible by a monitor or a printed externally 
connected to the digital video camera 1. 

25 

Fig. 32 in the next shows an exemplary configuration of an 
image processing system applied with the present invention. The 
image processing system 501 is constructed from a video camera 502 
for taking a picture of a subject and producing a wide-DR image 
30 L constructed from pixels having pixel values (luminance) having 
a dynamic range wider than ordinary ones, an image processing 
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apparatus 510 for compressing the grayscale of the wide-DR image 
L generated by the video camera 502 into a grayscale range displayable 
by a display 511, and the display 11 for displaying a 
grayscale-compressed image Lu generated by the image processing 
5 apparatus 510. 

The video camera 502 is constructed from a lens 503 for 
condensing a photo-image of a subject, a stop for adjusting an amount 
of light energy of the photo-image, a CCD image sensor for generating 

10 luminance signals by photo-electric conversion of the condensed 
photo-image, a pre-amplif ier (Pre-amp.) 506 for removing noise 
component from the generated luminance signals, an AD converter 
(A/D) 507 for converting the luminance signals, removed with the 
noise component, typically into digital data having a bit width 

15 of 14 to 16 bits or around, and an I/O interface (I/O) 508 f or outputting 
the wide-DR image, constructed from pixels having digitized 
luminance, to the image processing apparatus 510. 

Fig. 32 shows overall operations of the image processing system 
20 1. In step SlOl, the video camera 502 takes a picture of the subject, 
generates a correspondent wide-DR image L, and outputs it to the 
image processing apparatus 510. In step S102, the image processing 
apparatus 510 subjects the wide-DR image L to the grayscale 
compression processing to thereby generate the grayscale-compressed 
25 image Lu/ and outputs it to the display 511 . InstepS103, the display 
511 displays the grayscale-compressed image Lu. 

Fig. 34 in the next shows a first exemplary configuration of 
the image processing apparatus 510. Atone curve correction section 
30 521 of the image processing apparatus 510 corrects the wide-DR image 
L received form the video camera 502 in the direction of compressing 



the grayscale based on a preliminarily-obtained tone curve, and 
outputs the resultant tone-curve-corrected image Lc to a smoothed 
luminance generation section 522, a gain value setting section 523, 
and a contrast correction section 524. It is to be noted that the 
tone-curve-corrected image Lc has a compressed grayscale, and a 
weakened contrast ascribable to the compressed grayscale. The tone 
curve correction section 521 also outputs the representative value 

y, which expresses a slope of the tone curve used for the correction, 
to the gain value setting section 523. 

Fig. 35 shows an exemplary configuration of the tone curve 
correction section 521 . A LUT memory 531 of the tone curve correction 
section 521 preliminarily holds a lookup table (referred to as LUT, 
hereinafter) which corresponds with a monotonously-increasing tone 

curve as shown in Fig . 36, and a representative value y which expresses 
a slope of the tone curve. It is also allowable that a function 
corresponded to the tone curve is held in place of the LUT. A table 
reference section 532 corrects the wide-DR image L based on the 
LUT held by the LUT memory 531 to thereby obtain the 
tone-curve-corrected image Lc- 

Fig. 36 shows an example of the tone curve, wherein the abscissa 
plots luminance of the wide-DR image L, and the ordinate plots 
luminance of the tone-curve-corrected image Lc after the correction, 
respectively on the logarithmic axes normalized over a range of 
[0, 1] . The tone curve shown in Fig. 36 does not correct the luminance 
value of the normalized wide-DR image when the value exceeds 0.5, 
but corrects the luminance value of the normalized wide-DR image 
value when the value is smaller than 0.5 so that a smaller value 
is corrected by a larger amount of correction. That is, the 
correction is carried out so as to avoid maximum shadowing of the 



75 



dark area in the image when appeared on the display 511. The 

representative value y which expresses a slope of the tone curve 
can be defined by an average value of slopes respectively found 
over the entire regions of the luminance. For example, the 
5 representative value of the tone curve shown in Fig. 36 is 7=0.94. 

Ref erring now back to Fig. 34, the smoothed luminance 
generation section 522 smoothes the luminance of the 
tone-curve-corrected image Lc, and outputs the luminance Lci(p) of 
10 the obtained smoothed image to the contrast correction section 24. 
Fig. 37 shows an exemplary configuration of the smoothed luminance 
generation section 22 . 

A reduced image generation section 541 of the smoothed 
15 luminance generation section 522 classifies pixels of the 
tone -curve -corrected image Lc received from the tone curve correction 
section 521 into m^n blocks depending on positions of the pixels, 
and generates the reduced image Lci having an average value of the 
luminance of the pixels classified into the individual blocks as 
20 its own pixels. A reduced image memory 542 holds thus generated 
mxn-pixel reduced image Lci . An interpolation section 54 3 calculates , 
by interpolation, the luminance of the pixel position sequentially 
specified using the pixels of the reduced image held by the reduced 
image memory 542, and outputs the obtained interpolation value Lci (p) 
25 as luminance of the pixels of the smoothed image to the contrast 
correction section 524. It is to be noted herein that p=(x,y) is 
a coordinate or vector expressing the pixel position. Size of the 
smoothed image output from the interpolation section 54 3 is 
equivalent to that of the tone-curve-corrected image Lc. 

30 

That is, in the smoothed luminance generation section 522, 
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the tone-curve-corrected image Lc is reduced to generate the reduced 
image Lci, and by using the held reduced image Lci/ the luminance 
of the smoothed image is calculated by interpolation operation in 
a pixel-wise manner. 

5 

Although it was conventionally necessary to adopt a relatively 
large-scale filtering for an effective grayscale compression 
processing, the smoothed luminance generation section 522 needs 
only the reduced image memory 542 for holding the mxn-pixel reduced 
10 image. 

Fig. 38 shows an exemplary configuration of the reduced image 
generation section 541 shown in Fig. 37. A sorting section 551 of 
the reduced image generation section 541 sorts the pixels of the 

15 tone-curve-corrected image Lc received from the tone-curve 
correction section 521 into mxn blocks depending on the pixel position, 
and then supplies them to average value calculating sections 552-1 
to 552-N (=mxn) . For example, those classified into the first block 
are supplied to average value calculation section 552-1, and those 

20 classified into the second block are supplied to the average the 
value calculation section 552-2. The following description adopts 
a simple notation of average value calculation section 552 when 
there is no need of discrimination of the individual average value 
calculation sections 552-1 to 552-N. 

25 

The average value calculation section 552-i (i=l, 2, N) 
calculates an average value of the luminance of the pixels of the 
tone-curve-corrected image Lc classified into the i-th block, and 
outputs it to a composition section 553. The composition section 
30 553 generates an mxn-pixel reduced image logLci having, as a pixel 
value, the average value of the luminance respectively received 
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from the average value calculation means 552-i. 

Fig. 39 shows an exemplary configuration of the average value 
calculation section 552 shown in Fig . 38 • An adder 561 of the average 
5 value calculation section 552 adds the luminance of the 
tone-curve-corrected image Lc received from the sorting section 551 
in the preceding stage to a value held by a register (r) 562, to 
thereby update the value held by the register (r) 562. A divider 
563 divides a value finally held by the register 562 by the number 
10 of pixels Q composing one block, to thereby calculate an average 
value of the luminance of Q pixels classified into one block. 

Fig. 4 0 shows an exemplary configuration of the interpolation 
section 543 shown in Fig. 37. A vicinity selection section 571 of 
15 the interpolation section 543 acquires, upon reception of the 
position of interpolation p, 4x4-pixel luminance a [4] [4] in the 
vicinity of the position of interpolation p, based on the mxn-pixel 
reduced image logLci held by the reduced image memory 542, and outputs 
it to the products summation section 574. 

20 

A notation of a[i] [j] herein means that pixel value a is an 
ixj two-dimensional arrangement data. The vicinity selection 
section 571 outputs horizontal displacement dx and vertical 
displacement dy between the acquired luminance a [4] [4] and position 
25 of interpolation p to a horizontal coefficient calculation section 
572 or a vertical coefficient calculation section 573, respectively. 

It is to be noted that relations among the position of 
interpolation p, neighboring luminance a[4][4] and amounts of 
30 displacement dx, dy are similar to those described in the above 
referring to Fig . 11, sothattheexplanationthereforwillbe omitted. 
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The horizontal coefficient calculation section 572 calculates 
a tertiary interpolation coefficient k:x[4] in the horizontal 
direction based on the horizontal displacement dx received from 
5 the vicinity selection section 71. Similarly, the vertical 
coefficient calculation section 573 calculates a tertiary 
interpolation coefficient ky[4] in the vertical direction based on 
the vertical displacement dy received from the vicinity selection 
section 571. 

10 

The tertiary interpolation coefficient kx [ 4 ] in the horizontal 
direction is typically calculatedby using the equation (1) described 
in the above. 

15 The tertiary interpolation coefficient k:y[4] in the vertical 

direction is typically calculated by using the equation (2) described 
in the above. 

It is to be noted that any arbitrary calculation formula other 
20 than the equations (1) , (2) shown in the above may be used for the 
calculation of the tertiary interpolation coefficients kx[4] and 
ky[4] so far as a sufficiently smooth interpolation can be obtained. 

The products summation section 57 4 calculates an interpolation 
25 value Lci(p) of the position of interpolation p of the reduced image 
Lci by sum-of -products calculation using the neighboring pixel value 
a [4] [4], horizontal interpolation coefficient kx[4] and vertical 
interpolation coefficient ky[4], using the equation (3) described 
in the above. 

30 

Referring now back to Fig. 34, the gain setting section 523 
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calculates the gain value g(p), which is used for adjusting the 
amount of correction of the contrast of the luminance logLc(p) of 
the smoothed image in the contrast correction section 524, for the 

individual pixel positions based on the representative value y 
5 received from the tone curve correction section 521, and outputs 
it to the contrast correction section 524. 

The gain value g (p) will be explained below. For a gain value 
of g{p)=l, contrast is not enhanced nor suppressed by the contrast 
10 enhancement section 524. For a gain value of g(p)>l, contrast is 
enhanced corresponding to the value. On the contrary, for a gain 
value of g(p)<l, contrast is suppressed corresponding to the value. 

Outline of the gain value setting by the gain setting section 
15 523 is similar to the gain value setting by the above-described 
gain setting section 93, so that the explanation therefor will be 
omitted. 

Fig. 41 shows an exemplary configuration of the gain value 
20 setting section 523. A divider 581 calculates the inverse l/Y=go 
of the representative value Y received from the preceding stage, 
and outputs it to a subtracter 582. The subtracter 582 calculates 
(go-1) and output it to a multiplier 588. 

25 A subtracter 583 calculates difference (Lc-Lgray) between the 

individual luminance of the tone-curve-corrected image Lc and the 
luminance Lgray having a moderate gray level, and outputs it to a 
divider 585. The subtracter 584 calculates difference (Lwhite"Lgray) 
between the luminance Lwhite having a white clipping level and luminance 

30 Lgrayf and outputs it to a divider 585. The divider 585 divides the 
output (Lc-Lgray) of the Subtracter 583 by the output (Lwhite-Lgray) 
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of the subtracter 584, and outputs it to an absolute value calculator 

586 . The absolute value calculator 586 calculates an absolute value 
of the output from the subtracter 585, and outputs it to a clipper 

587. The clipper 587 clips the output from the absolute value 
5 ■ calculator 586 so as to adjust it to 1 when the output exceeds 1, 

but leaves it unchanged when the output does not exceed 1, and outputs 
the result as attn(p) to a multiplier 588. 

The multiplier 588 multiplies the output from the subtracter 
10 582 by the output from the clipper 587, and outputs the product 
to an adder 58 9. The adder 58 9 adds 1 to the output from the multiplier 

588. and outputs the result as the gain value g(p) to the succeeding 
stage . 

Referring now back to Fig. 34, the contrast correction section 
524 enhances the contrast of the tone-curve-corrected image Lc which 
has previously been weakened, based on the gain value g(p) for the 
individual pixel positions p received from the gain value setting 
section 523 and the luminance Lci(p) of the smoothed image received 
from the smoothed luminance generation section 522, to thereby 
generate the grayscale-compressed image Lu. 

Fig. 42 shows an exemplary configuration of the contrast 
correction section 524 . A subtracter 591 of the contrast correction 
25 section 524 calculates difference (Lc (p) -Lci (p) ) between the 
luminance Lc (p) of the individual pixels of the tone-curve-corrected 
image Lc and the luminance of the corresponded pixels of the smoothed 
image (i.e, interpolation value Lci(p) of the reduced image), and 
outputs it to a multiplier 592. The multiplier 592 calculates a 
30 product of an output of the subtracter 591 and the gain value g(p) 
received from the gain value setting section 523, and outputs the 



15 



20 
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result to an adder 593. The adder 593 adds the luminance of the 
smoothed image (interpolation value Lci{p) of the reduced image) 
to an output of the multiplier 592 , and outputs the obtained luminance 
Lu(p) / as the luminance of pixels composing the contrast-corrected, 
5 grayscale-compressed image Lu, to the succeeding stage. 

It is to be noted now that the luminance of pixels of the smoothed 
image (interpolation value Lci(p) of the reduced image) is an 
interpolated value based on the pixels of the mxn-pixel reduced 
10 image Lci/ and therefore has only an extremely-low-frequency 
component of the image Lc before being reduced. 

Hence, the output (Lc(p)- Lci(p)) of the subtracter 591 is 
equivalent to that obtained by subtracting only the 

15 extremely-low-frequency component from the original 
tone-curve-corrected image logLc- The luminance Lu(p) of the 
contrast-corrected, grayscale-compressed image is such as being 
obtained, as described in the above, by dividing the luminance signal 
into the extremely-low-frequency component and other components, 

20 and of these, the components other than the low-frequency components 
(output of the subtracter 591) are enhanced in the contrast by being 
multiplied by the gain value g(p), and by again synthesizing the 
both using the adder 593. 

25 As described in the above, the contrast correction section 

524 is configured so as to enhance the components in the 
low-to-middle-frequency region and high-frequency region, except 
for the extremely-low-frequency region, using the same gain value 
g(p) . Therefore, it is made possible to obtain an image having the 

30 contrast enhanced very naturally to the eyes, without generating 
a local overshoot of the edge portion which may otherwise be distinct 
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when only the high-frequency component is enhanced. 

Next, the grayscale-compressed image generation processing 
by the image processing apparatus 510 according to the first exemplary 
5 configuration (i.e., the processing in step S102 described in the 
above referring to the flow chart in Fig. 33) will be explained 
referring to the flow chart in Fig. 43. 

In step Sill, the tone curve correction section 521 corrects 
10 the luminance of the wide-DR image L received from the video camera 
502 based on the preliminarily-obtained LUT , and outputs the obtained 
tone-curve-corrected image Lc to the smoothed luminance generation 
section 522, gain value setting section 523, and contrast correction 
section 524. The tone curve correction section 521 also outputs 

15 the representative value y, which expresses a slope of the tone curve 
used for the correction, to the gain value setting section 523. 

In step S112, the smoothed luminance generation section 522 
shrinks the tone-curve-corrected image Lc to thereby generate the 
20 reduced image Lci^ and further calculates the luminance Lci(p) of 
pixels of the smoothed image based on the interpolation operation 
using the pixels of the reduced image Lci, and outputs the result 
to the contrast correction section 524. 

25 In step S113, the gain setting section 523 calculates the gain 

value g(p) which is used for adjusting the amount of correction 
of the contrast of the luminance Lc(p) of the smoothed image in the 
contrast correction section 524, for the individual pixel positions 

based on the representative value y received from the tone curve 
30 correction section 521, and outputs it to the contrast correction 
section 524. 
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It is to be noted that the processing in step S112 and step 
S113 can be carried out in parallel. 

5 In step S114, the contrast correction section 524 corrects 

the luminance of the tone-curve-corrected image Lc/ based on the 
gain value g(p) for the individual pixel positions p received from 
the gain value setting section 523, and the luminance Lci(p) of the 
smoothed image received from the smoothed luminance generation 

10 section 522, to thereby calculate the luminance Lu(p) of pixels of 
the grayscale-compressed image Lu. The contrast-corrected 
grayscale-compressed image Lu that is obtained as described above 
is therefore obtained as an image having the contrast enhanced very 
naturally to the eyes, without generating a local overshoot of the 

15 edge portion which may otherwise be distinct when only the 
high-frequency component is enhanced. Here is the end of the 
explanation on the grayscale-compressed image generation processing 
by the first exemplary configuration of the image processing 
apparatus 510. 

20 

Fig. 44 in the next shows a second exemplary configuration 
of the image processing apparatus 510. The second exemplary 
configuration is such as being configured so that a logarithmic 
conversion section 601 for effecting the logarithmic conversion 

25 of the luminance of the wide-DR image L received from the video 
camera 501 is provided in the preceding stage of the tone curve 
correction section 521 in the first exemplary configuration shown 
in Fig. 34, and so that a logarithmic inversion section 602 for 
effecting the logarithmic inversion of the output of the contrast 

30 correction section 524 is provided in the succeeding stage of the 
contrast correction section 524 in the first exemplary 
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configuration . 

Any constituents other than the logarithmic conversion section 
601 and logarithmic inversion section 602 composing the second 
5 exemplary configuration of the image processing apparatus 510 are 
equivalent to those in the first exemplary configuration shown in 
Fig. 34, and are given with the same reference numerals, so that 
the explanations therefor will be omitted. It is to be noted herein 
in the second exemplary configuration that the sections from the 
10 tone curve correction section 521 to the contrast correction section 
524 individually processes the luminance after logarithmic 
conversion. 

The tone curve correction section 521 in the second exemplary 
15 configuration adopts the tone curve such as shown in Fig. 4, for 
example. Application of the monotonously-increasing moderate 
inverse-S-shaped curve as shown in Fig. 4 will not give so strong 
effect of grayscale compression in the high luminance region and 
low luminance region, so that it is possible to obtain a desirable 
20 tone with less degree of whiteout or blackout even after the grayscale 
compression. On the contrary, the grayscale compression will 
strongly affect the middle luminance region, but this means that 
the contrast correction can fully be applied with the middle luminance 
region, and results in a desirable grayscale-compressed image Lu 
25 with a less degree of contrast correction also in the middle luminance 
range. The tone curve herein has a representative value y of 0.67. 

Next, details of the grayscale-compressed image "generation 
processing according to the second exemplary configuration of the 
30 image processing apparatus 510 will be explained referring to the 
flow chart in Fig. 45. 
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In step S121, the logarithmic conversion section 601 subjects 
the wide-DR image L received from the video camera 502 2 into 
logarithmic conversion, and outputs the obtained logarithmic wide-DR 
5 image logL to the tone curve correction section 521. 

In step S122, the tone curve correction section 521 corrects 
the luminance of the logarithmic wide-DR image logL, typically based 
on the preliminarily-obtained LUT, which corresponds to the tone 
10 curve shown in Fig. 4, and outputs the obtained logarithmic 
tone-curve-corrected image logLc to the smoothed luminance 
generation section 522, gain value setting section 523, and contrast 
correction section 524. The tone curve correction section 521 also 

outputs the representative value y, which expresses a slope of the 
15 tone curve used for the correction, to the gain value setting section 
523. 

In step S123, the smoothed luminance generation section 522 
shrinks the logarithmic tone-curve-corrected image logLc to thereby 
20 generate the logarithmic reduced image logLci, and further calculates 
the luminance logLci(p) of pixels of the logarithmic smoothed image 
by the interpolation operation using pixels of the logarithmic 
reduced image logLci/ and outputs the result to the contrast correction 
section 524. 

25 

In step S124, the gain setting section 523 calculates the gain 
value g(p) which is used for adjusting the amount of correction 
of the contrast of the luminance logLc (p) of the logarithmic smoothed 
image in the contrast correction section 524, for the individual 

30 pixel positions based on the representative value y received from 
the tone curve correction section 521, and outputs it to the contrast 
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correction section 524. 

It is to be noted that the processing in step S123 and step 
S124 can be carried out in parallel. 

5 

In step S125, the contrast correction section 524 corrects 
the luminance of the logarithmic tone-curve-corrected image logLc 
based on the gain value g(p) for the individual pixel positions 
p received from the gain value setting section 523, and the luminance 
10 logLci (p) of the logarithmic smoothed image received from the smoothed 
luminance generation section 522, to thereby calculate the luminance 
logLu(p) of pixels of the logarithmic grayscale-compressed image 
logLu, and outputs it to the logarithmic inversion section 602. 

15 In step S126, the logarithmic inversion section 602 subjects 

the luminance logLu(p) of pixels of the logarithmic 
grayscale-compressed image logLu to the logarithmic inversion, and 
outputs the obtained Lu(p) as the luminance of pixels of the 
grayscale-compressed image Lu- 

20 

Because the contrast-corrected grayscale-compressed image Lu 
that is obtained as described above will not give so strong effect 
of grayscale compression in the high luminance region and low 
luminance region, so that it is possible to obtain a desirable tone 

25 with less degree of whiteout or blackout even after the grayscale 
compression. On the contrary, the grayscale compression will 
strongly affect the middle luminance region, but this means that 
the contrast correction can fully be applied with the middle luminance 
region, and results in a desirable ' grayscale-compressed image Lu 

30 with a less degree of contrast correction also in the middle luminance 
range . Here is the endof the explanationof the grayscale-compressed 
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image generation processing according to the second exemplary 
configuration of the image processing apparatus 510. 

As has been described in the above, the image processing 
5 apparatus 510 according to one embodiment of the present invention 
makes it possible to convert a wide-DR image, having a dynamic range 
of luminance wider than the usual, to a grayscale-compressed image 
displayable on the display 111 having only a narrow dynamic-range 
of displayable luminance, based on a configuration largely reduced 

10 in a large capacity of memory (used as a frame memory and data delay 
line) which has been indispensable for the conventional grayscale 
compression technique, without ruining the nice-looking property- 
It is also made possible to obtain a grayscale-compressed image 
which is by no means inferior to that obtained by the grayscale 

15 compression processing conventionally been realized using a 
large-scale filtering . 

Of course, the image processing apparatus 510 can convert the 
wide-DR image into the grayscale-compressed image, while being 
20 adapted to the dynamic range expressible on printers and pro j ectors, 
besides the display 511. 

The present invention is typically applicable to image signal 
processing circuit incorporated not only in photographing 
25 apparatuses such as digital video camera and digital still camera, 
but also in expression apparatuses such as display, printer, 
projector and so forth. 

A series of processing described in the above can be executed 
30 on the hardware basis, but can be executed also on the software 
basis- For the case where the series of processing is executed on 
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the software basis, a program composing the software is installed 
from a recording medium on a computer incorporated in a dedicated 
hardware, or, for example, to a general-purpose personal computer 
which can execute a variety of functions after being installed with 
5 a variety of programs. 

Fig. 4 6 shows an exemplary configuration of a general-purpose 
personal computer. The personal computer 620 has a CPU (Central 
Processing Unit) 621 incorporated therein. The CPU 621 is connected 
10 with an input/output interface 625 via a bus 624. The bus 624 is 
connected with a ROM (Read Only Memory) 622 and a RAM (Random Access 
Memory) 623. 

The input/output interface 625 is connected with an input 
15 section 62 6 which is constructed from an input apparatus such as 
keyboard, mouse and so forth through which the user enters operational 
commands, an output section 627 for outputting processing operation 
screen or resultant image of processing on a display apparatus, 
a storage section 628 structured with a hard disk drive and so forth 
20 for storing programs of various data, and an I/O interface 629 used 
for image data communication with the video camera 502 or the like. 
It is also connected with a drive 630 used for writing/reading of 
data to or from a recording medium such as a magnetic disk 631 
(including flexible disk), optical disk 632 (including CD-ROM 
25 (Compact Disc-Read Only Memory) and DVD (Digital Versatile Disc) ) , 
a magneto-optical disk 633 (including MD (Mini Disc) ) , or a 
semiconductor memory 634. 

The CPU 621 executes various processing according to a program 
30 stored in the ROM 622, or a program read out from any of the media 
ranging from the magnetic disk 631 to semiconductor memory 634 and 
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installed on the storage section 628, and loaded from the storage 
section 628 to the RAM 623 from the storage section 628. The RAM 
623 has, properly stored therein, also data necessary for the CPU 
621 to carry out various processing. 

5 

It is to be understood that, in this patent specification, 
the steps for describing the program recorded in a recording medium 
includes not only processing carried out according to the given 
order on the time-series basis, but may also contain processing 
10 carried out in parallel or individually in place of those always 
processed on the time-series basis. 

It is to be understood that the whole contents of the claims, 
specifications, drawings and abstracts of Japanese Patent 
15 Application No. 2002-003134 (filed on January 9, 2003) and No. 
2003-003135 (filed on January 9, 2003) are referred to and 
incorporated herein in this specification. 

INDUSTRIAL APPLICABILITY 

20 

As has been described in the above, the present invention makes 
it possible to realize a grayscale compression technique which 
requires only a small memory capacity to be consumed and a light 
load of calculation, allows easy hardware construction, and ensures 
25 a large grayscale compression effect. 

It is also made possible to appropriately enhance the contrast 
of image using a smaller capacity of memory, based on a less amount 
of calculation, and based on a simple hardware construction- 



